Last Friday, June 14th, members of The Information Lab Ireland’s consulting and L&D teams attended Data Céilí, a community-lead conference with a focus on Microsoft Data Platform technologies. The free event took place in the Arts Block at Trinity College Dublin where attendees enjoyed a day jam-packed with diverse session tracks and great networking opportunities with peers and technology partners.
Our team in attendance included Kevin Lynch, Louise Shorten, Gary Vance and Tom Connolly. Their highlights of the event included:
meeting prominent Microsoft learning influencers
the community-led feel
the opportunity to share ideas and learnings with other like-minded data professionals
gaining insights into Microsoft's rapidly innovating cloud data technologies.
The Team’s Key Takeaways from Data Célií:
The Size of the Microsoft Ecosystem: Sometimes, we can become siloed in our own cocooned areas of Power BI or Data Engineering. However, when you take a step back and include elements like Fabric, OpenAI, Power Platform, the wider Azure Platform, and CoPilot, an event like this helps to open up the realms of the possible by giving us a full view of this ecosystem, which is huge!
Fabric's Growing Impact: It’s clear that Fabric is quickly becoming the biggest tool in the data landscape. It has shortcuts, connectors, and mirroring with almost every other data tool out there, making data integration within Fabric incredibly simple. The Advancing Analytics team showcased a scenario of using both Databricks and Fabric together, utilising the medallion architecture. This was a great session full of insights and knowledge.
AI and ML Integration: AI and ML are very much here and now. Whether it's the use of Co-pilot throughout the Microsoft ecosystem or self-built LLM tools through OpenAI and Azure AI to AI visuals and What-If parameters in Power BI, the ability to implement AI and ML is now within touching distance for any company.
Fabric Warehouse: We can see this platform become a key player in the data warehousing space over the coming months. Many of us already have heard plenty about Fabric, Microsoft's end to end data analytics offering. Within that, is Fabric Warehouse, the analytical data warehouse offering. Fabric warehouse is optimised for OLAP (online analytical processing), and competing in the same space as data platforms like Snowflake. One of Fabric's key features is its adoption of the open source Delta (parquet based) format. This is really significant, because in doing so it has really opened things up in terms of interoperability across other platforms (Databricks, Snowflake and others). It was interesting to hear about the features planned for release in the near future and about its seamless integration to the Microsoft ecosystem.
GreenOps: GreenOps, focusing on optimising and reducing CO2 costs, was highlighted in Henk Van der Valk's talk, offering practical suggestions such as choosing regions with higher renewable energy adoption for infrastructure deployment. While storage is generally seen as less costly than compute, the potential for optimisation in big data environments is significant, impacting both costs and CO2 emissions. Key strategies include minimising unnecessary logging and metadata and selecting appropriate storage tiers. For further learning, numerous sustainability groups and foundations like greensoftware.foundation, cncf.io, and sustainability.org provide valuable resources online and on LinkedIn.
Fabric Direct Lake: If you are not familiar with it already, you can think of Direct Lake as a semantic model capability in Power BI, but for very large data volumes. It allows Power BI to load Parquet files directly from a data lake, effectively bypassing the need to query a lakehouse or warehouse endpoint or import and duplicate data into the Power BI model. Seeing this in action in a demo, and how simple this was to set up, was key. It is this level of interoperability, and integration across so many complementary tools and services that really sum up what Microsoft and Azure are bringing to the table, across the board. Hugely impressive.
The Team’s Favourite Data Célií Sessions:
Tabular Editor: Microsoft Most Valuable Professional (MVP) Sue Bayes spoke with passion in her talk about Tabular Editor. This is an external product which can make the life of a Power BI developer easier and allow them to work more efficiently. One of the advantages highlighted was the ability to run macros to modify and add fields with ease across multiple tables and models. There is added transparency around dependencies and benefits in terms of documenting work and even the ability to check your data model against best practice guidelines. Here is a link to her resources on Github: https://github.com/databayes/Session_Resources
Co-pilot for Power Apps This was a great talk delivered by Foyin Olajide-Bello. Her command of Power Apps was evident and the demo showed how people new to Power Apps could leverage the Co-pilot to add basic functionality. She also showed how users can explore some of the more complex features using the explanations provided. The wider discussion touched on the subject of reverse prompting when leveraging generative AI and was a very interesting session overall.
Power BI Composite and Advanced Models: This talk was delivered by renowned YouTube teacher Alex Whittles (Power BI Sentinel). The session was very informative, and even after being a Power BI user for 6+ years, I was able to learn some new things about the various types of models in Power BI. It’s great to now be able to recognise which type of model to use in a given scenario.
Leading High Performing Teams: The session delivered by John Martin on leading high performing teams provided valuable food for thought and complemented some of the more technical sessions. He got the audience thinking about the qualities of a successful data leader in an ever evolving technical landscape and talked about the importance of aligning team growth with the business needs. There was a focus on designing frameworks and methods for measuring work and tracking progress.
Conclusion
Thumbs up all-around from our team, who had a highly informative day at Data Céilí and took home lots of learnings and insights to share with both customers and colleagues. The sessions covered a range of topics that are directly applicable to our work, enhancing our understanding of the latest advancements in Microsoft's data platform technologies.
Feel free to touch base with us to discuss any of the insights we gained or to explore the exciting developments we learned about at the conference.