Configure Power BI Lineage Collection without Log Analytics (Beta)

BETA FEATURE: BI Optimizer is a beta feature. Functionality, configuration steps, and supported options are subject to change as the feature evolves.

NOTE: We recommend configuring Log Analytics for Lineage to improve the precision of Lineage metadata collection. With Log Analytics enabled, Lineage returns more accurate data on columns and tables from which the Reports are derived. In case Log Analytics is disabled, the metadata returned from the Power BI available in Lineage may contain duplicates, such as extra tables and columns.

This article explains how to configure Microsoft Power BI Cloud Data Source for Lineage collection. This configuration is required for enabling BI Optimizer in Metric Insights.

The instructions are intended for the Metric Insights administrator and assume that the configuration in Azure has been completed by your organization's Azure administrator, and you have been provided with the configured Azure application credentials. These credentials will be used to set up Data Source in Metric Insights.

Before proceeding, ensure that all prerequisites are met and that the initial configuration for your selected authentication type is complete. Then enable metadata collection and configure the Power BI Cloud Data Source, set up Object List Refresh, and sync metadata.

PREREQUISITES:

  1. Set up Azure for Power BI Lineage Collection.

TABLE OF CONTENTS:

  1. Enable Metadata Collection in Metric Insights
  2. Configure Power BI Cloud Data Source in Metric Insights
    1. Create a Microsoft Power BI Cloud Data Source
    2. Configure Auth Parameters
    3. Configure Object List Refresh
      1. Manually Refresh Object List
      2. Add Object List Refresh Trigger
    4. Sync Metadata

1. Enable Metadata Collection in Metric Insights

Access Admin > System > System Variables

  1. Enter ENABLE_METADATA_COLLECTION in the search field.
  2. Click the gear icon.
  3. Assigned Value: Y.
  4. [Save], then [Commit Changes]

NOTE: See Setting System Variables for information on System Variables configuration.

2. Configure Power BI Cloud Data Source in Metric Insights

2.1. Create a Microsoft Power BI Cloud Data Source

See the instructions on creating a new Microsoft Power BI Cloud Data Source for your Metric Insights version:

2.2. Configure Auth Parameters

Proceed with the Data Source parameters configuration based on the selected Auth Type:

Expand or collapse content Service Principal
Expand or collapse content v7.2.0+
  1. Auth Type: Service Principal.
  2. Application ID: Enter the Application (client) ID of the Azure application.
  3. Client Secret: Enter the Client Secret Value of the Azure application.
  4. Directory (tenant) Id: Enter Directory (tenant) Id of the the Azure application.
Expand or collapse content Prior to v7.2.0
  1. Auth Type: Service Principal.
  2. Application ID: Enter the Application (client) ID of the Azure application.
  3. Client Secret: Enter the Client Secret Value of the Azure application.
  4. Server: https://app.powerbi.com.
  5. Directory (tenant) Id: Enter Directory (tenant) Id of the the Azure application.
Expand or collapse content Password
Expand or collapse content v7.2.0+
  1. Auth Type: Password.
  2. Username, Plugin Password: Enter the credentials of a Fabric Administrator user.
  3. Application ID: Enter the Application (client) ID of the Azure application.
Expand or collapse content Prior to v7.2.0
  1. Auth Type: Password.
  2. Application ID: Enter the Application (client) ID of the Azure application.
  3. Server: https://app.powerbi.com.
  4. Username, Plugin Password: Enter the credentials of a Fabric Administrator user.
Expand or collapse content OAuth
Expand or collapse content v7.2.0+
  1. Auth Type: OAuth.
  2. Application ID: Enter Application (client) ID of the Azure application.
  3. Client Secret: Enter Client Secret Value of the Azure application, if it has been generated.
  4. Directory (tenant) Id: Enter Directory (tenant) Id of the of the Azure application.
  5. [Get Token], enter credentials of a Fabric Administrator in the pop-up window.
Expand or collapse content Prior to v7.2.0
  1. Auth Type: OAuth.
  2. Application ID: Enter Application (client) ID of the Azure application.
  3. Client Secret: Enter Client Secret Value of the Azure application, if it has been generated.
  4. Server: https://app.powerbi.com.
  5. Directory (tenant) Id: Enter Directory (tenant) Id of the of the Azure application.
  6. [Get Token], enter credentials of a Fabric Administrator in the pop-up window.

2.3. Configure Other Data Source Parameters

See parameter configuration details specific to your MI version:

Expand or collapse content v7.2.0

Power BI Object Types must include Dashboards and Reports.

Expand or collapse content v7.1.1-v7.1.2a

Scope: Types to collect must include DASHBOARD, REPORT, and LOG.

Expand or collapse content Prior to v7.1.1

Object types to collect must include DASHBOARD, REPORT, and LOG.

2.4. Configure Object List Refresh

2.4.1. Manually Refresh Object List

Access Source Objects tab

  1. [Refresh list]

2.4.2. Add Object List Refresh Trigger

Adding an Object List Refresh Trigger allows to automatically refresh the Object List at the configured time interval.

See the instructions for your MI version:

Expand or collapse content v7.2.0+

Access the Advanced tab

  1. Generate Object List: Automatically.
  2. Object List Refresh Trigger: Select a daily Trigger.
Expand or collapse content Prior to v7.2.0
  1. Generate Object List: automatically.
  2. Object List Refresh Trigger: Select a daily Trigger.

2.5. Sync Metadata

NOTE: Metadata sync is not related to the previously configured Object List Refresh Trigger and needs to be updated manually on the regular basis.

See instructions on syncing Metadata for your MI version:

Expand or collapse content v7.2.0+

Beginning in v7.2.0, Metadata can be synced from either of the following locations:

  1. From the Metadata tab: click [Sync Metadata]
  2. From any tab in the Plugin Data Source Editor: click [Refresh Metadata] in the top panel.
Expand or collapse content Prior to v7.2.0

Access Metadata tab

  1. [Sync Metadata]