Create Dataset with Concierge (Beta)

BETA FEATURE: This is a beta feature introduced in v7.2.1. It can change as the feature evolves.

When using Concierge, users can create Datasets not by collecting data from an external tool, but by analyzing Datasets that already exist in Metric Insights or by gathering various types of publicly available information. It also allows users to write a prompt instead of an SQL query to populate the Dataset with values.

Use this detailed step-by-step guide to create a Dataset using Concierge.

PREREQUISITES:

Create a Concierge Data Source

NOTE: Please note that using Concierge as a Data Source involves transmitting data to Concierge and, subsequently, to the LLM. Exercise appropriate caution when handling sensitive information.

1. Add New Dataset

Access +New >Dataset > Create New

  1. Measured: Select the measurement interval that applies to the level of aggregation that you want in your result set.
  2. Name: Give the Dataset a descriptive name.
  3. Category: Specify the Category to place the Dataset.

2. Complete the Data Tab

  1. Data Source: Select the Concierge Data Source.
  2. Trigger: Specify the Data Collection Trigger that will be used to collect data for the Dataset.
  3. Domain: Select the Domain that will serve as the basis for the Dataset.
    • Concierge won't send queries to any External Resources unless the corresponding Domain is selected here. The LLM Domain is a general-use Domain, best suited for Public Data Datasets.
  4. Prompt: Write the prompt as you normally would for Concierge, describing what should be placed into the Dataset.
    • The prompt must contain a Format section, specifying which columns the Dataset should contain and what type of values to place in them.
  5. [Validate]

2.1. Public Data Dataset

If you want to create a Dataset from publicly available information, such as weather forecasts or currency values, add a Data Web Endpoint from the website you wish to collect data from. Websites offering open public data typically have endpoints listed for users.

You must also enable the LLM_ALLOWED_DOMAINS System Variable.

2.1.1. Enable System Variable

Access Admin > System > System Variables

  1. Find LLM_ALLOWED_DOMAINS Variable and click the gear icon to open the Edit Variable window.
  2. Assigned value: List all domains with public data you would be using.
  3. [Save]
  4. [Commit Changes]

For more information on how to work with System Variables, refer to Setting System Variables.

2.2. Analytics Dataset

If you need Concierge to analyze existing Datasets, you must mention them in the prompt and then enable their usage in the separate editor.

Reference Datasets via dataset_id=#####.

2.2.1. Enable Datasets

[Save] the Dataset and open the Editor by typing the [instance]/editor/llm/dataset/[dataset_id] into the browser bar.

  1. [+ New Dataset]
  2. Select the Dataset(s) from the list.
  3. [Save]

By doing this you allow the Concierge to analyze data stored in those Datasets and use it to create a more specific Dataset.

3. Save and Enable Dataset

  1. [Save] the Dataset.
  2. [Enable & View] to enable the Dataset and open the Dataset Viewer.

4. Dataset Viewer

The Dataset Viewer lets you view data and create Dataset Views. For more details, see the Access, Create, and modify Dataset Views article.

You can also create a Dataset Report. For more information on creating Dataset Reports, refer to the Dataset Reports Overview article.