Microsoft Fabric Migration Consultancy

You are here

Microsoft Fabric BI Platform Development & Migration Strategy

Many organisations are now faced with a decision about whether to migrate to Microsoft Fabric for their preferred BI and Data Analytics platform. At the time of writing, December 2024, Fabric has been in production for 1 year, and during that year it has evolved into a powerful data analytics platform offering all BI services in a single integrated space.

As a platform it is still evolving, and does not currently have all of the functionality of other data analytics platforms, so it needs to be a careful decision to assess if now is the time for you to migrate your data analytics solutions to Microsoft Fabric, or whether you should wait a while.

Where is your BI solution now? 

  • You may currently have a Power BI based Business Intelligence solution serviced by an Azure SQL based Data Platform that has scaled rapidly as your business has grown over recent years
  • You may have an enterprise scale Azure Synapse data warehouse solution.
  • You may have a BI solution hosted entirely by the Power BI platform that is bursting at the seams and needing a formal data warehouse approach.
  • You may have a data lake, or may have thought about implementing a datalake.

Scalability, Load and Cost Implications

Whilst Azure SQL, Azure Synapse, Azure Data Factory and Power BI are very much still current and fit for purpose tools, and they are still functioning well and meeting requirements, they may have reached a point of load and cost where other tools may be more appropriate and it is sensible to be proactive in addressing this to ensure the platform can continue to scale with the business and its increasing appetite for data, whilst being mindful of budget.

Do you need to review your BI strategy?

A comprehensive review of your current situation and strategy is required to identify where more technically and/or financially appropriate tools are available to ensure scalability and reliability of the platform, along with licensing and cost implications. Where suitable changes are identified, an implementation plan will be required to ensure continuity of existing functionality whilst completing the migration in a timely fashion.

The key objectives might be:

  • Consolidate data maintenance procedures into a single orchestration tool.
  • Ensure long-term scalability and cost effectiveness of data warehouse/lake.
  • Take advantage of new and enhanced functionality of Power BI, Azure, and Fabric.
  • Revise modelling strategy to simplify model development, improve reliability, and support complex security requirements.
  • Retire components that can be consolidated into other services, such as Analysis Services.
  • Improve collaborative development experience and build robust DevOps practice.
  • Provide greater administrative control and monitoring.
  • De-risk change management process, particularly with regards shared datasets.

 

 

Microsoft Fabric Migration Scenario

The following diagram shows how an existing Azure Synapse or Azure SQL based BI solution might translate into a Microsoft Fabric BI solution:

 

Current BI Solution

Your current BI solution might employ the following components.

Unstructured Storage – Azure Data Lake Storage Account

Data Lake storage is used to consolidate file-based exports from some systems and synchronised copies of spreadsheets from SharePoint to make them accessible to Azure Data Factory for ingestion into the Data Warehouse.
This can be maintained as is, or could be integrated into Microsoft Fabric OneLake

Data Warehouse – Azure SQL Server

The data warehouse that serves the majority of reporting use cases is currently deployed using Azure SQL Server. This was chosen for its flexible scaling and pricing options that were appropriate for the volumes of data initially considered. 
It is not purpose built for analytics and has now reached a scale where it is likely to be more performant and cost effective to migrate to a purpose designed data warehouse platform such as Azure Synapse, or Fabric.

Orchestration – SQL Server Integration Services & Azure Data Factory

There are currently 2 orchestration tools in use to maintain the content of the Data Warehouse. 


SQL Server Integration Services (SSIS)

SSIS is a component of a traditional on-premises SQL Server deployment but in this case has been deployed to a cloud hosted virtual machine. SSIS was primarily chosen because of the requirement to make use of the ODBC connector for NetSuite. This required the installation of a specific driver which would have been relatively expensive and high maintenance to deploy through Azure Data Factory. As such, a Virtual Machine running SQL Server was a more cost-effective solution.

Azure Data Factory

For more recent data sources, Azure Data Factory has been used to implement ELT pipelines in order to provide greater visibility, simpler administration, and pursue a more general strategy away from Infrastructure-as-a-Service and towards more managed Platform-as-a-Service solutions. 
Both Data Factory and your application/ERP platforms may have had some considerable technology advances in recent years and months and it would be beneficial to revisit this process to consolidate orchestration into a single toolset.

Data Factory pipelines have also now been made available in the Power BI environment under Microsoft Fabric.

Semantic Modelling – Azure Analysis Services

The model provides a high-performance in-memory cache of the data for analysis, along with centralised definition of business logic, relationships, hierarchies, and calculations. 
Azure Analysis Services was initially chosen for this as license limitations of Power BI were too prohibitive at the time for some requirements, particularly with regards model size and refresh frequency.

Migrating the modelling process into Microsoft Fabric may now make larger more fast-moving models more accessible, and the complexity of some modelling requirements, particularly with regards security, have made maintaining a single monolithic model in Analysis Services too complex and vulnerable.
 

Proposed Solution

Microsoft Fabric can be described as a unified analytics platform is an evolution of several existing Microsoft analytics products, consolidated into a single administrative environment and licensing model which makes licensing, development, and administration much simpler. All the existing components of Power BI remain much as they are:

  • Dataflows – Automated data collection and transformation process
  • Datasets – Shared Semantic Models that can serve multiple reports
  • Reports – Data visualization

Fabric offers some valuable additional features that will be of great benefit to maintaining the existing Power BI components, including:

  • DevOps Integration – Version Control

    • Fabric workspaces can be integrated with Azure DevOps so datasets and reports can be checked in for version control and change management
  • Larger Model support

    • Fabric capacities can be used to maintain larger models (>1GB) whilst still enabling users to access this content under their standard Pro license (or no user license at all if capacity >F64)
  • Browser based development

    • All aspects of the Power BI process, including modelling and visualization, can now be accomplished entirely within the browser making development much more accessible and eliminating the need for desktop tools to be installed.

In addition to the enhancements to existing components, Fabric also incorporates many of the traditionally Azure based Data Platform and orchestration components in to the Fabric workspace environment, including:

  • OneLake – a unified, logical data lake for the entire organization

    • A common governance layer over workspaces that enables top down administrative visibility
    • Makes a single set of data accessible to a variety of toolsets without the need for duplication
  • Data Warehouse – high performance analytics

    • A fully functional Read/Write T-SQL database engine with distributed compute to support large scale analytics workloads
  • Data Lakehouse – rapid access to data

    • A virtual data warehouse that maintains parquet file based storage in OneLake with a read-only SQL endpoint to enable querying of the data directly within the lake.


How can we help?

PTR are a Microsoft Solutions Partner for AI & Azure Data, and providers of highly skilled technical consultancy and training services, specialising in Business Intelligence, Analytics and the Data Platform that serves these functions. We have implemented BI solutions for clients across a multitude of data platforms and services.


We are well versed in providing high level technology recommendations  and data strategy review as along with cost implications, and can offer support to implement both new BI projects and support Microsoft Fabric migration a projects.

Give us a call on 0118 979 4000 or email us at info@ptr.co.uk.