Importing and Exporting Datasources

The superset cli allows you to import and export datasources from and to YAML. Datasources include both databases and druid clusters. The data is expected to be organized in the following hierarchy:

├──databases
|  ├──database_1
|  |  ├──table_1
|  |  |  ├──columns
|  |  |  |  ├──column_1
|  |  |  |  ├──column_2
|  |  |  |  └──... (more columns)
|  |  |  └──metrics
|  |  |     ├──metric_1
|  |  |     ├──metric_2
|  |  |     └──... (more metrics)
|  |  └── ... (more tables)
|  └── ... (more databases)
└──druid_clusters
   ├──cluster_1
   |  ├──datasource_1
   |  |  ├──columns
   |  |  |  ├──column_1
   |  |  |  ├──column_2
   |  |  |  └──... (more columns)
   |  |  └──metrics
   |  |     ├──metric_1
   |  |     ├──metric_2
   |  |     └──... (more metrics)
   |  └── ... (more datasources)
   └── ... (more clusters)

Exporting Datasources to YAML

You can print your current datasources to stdout by running:

superset export_datasources

To save your datasources to a file run:

superset export_datasources -f <filename>

By default, default (null) values will be omitted. Use the -d flag to include them. If you want back references to be included (e.g. a column to include the table id it belongs to) use the -b flag.

Alternatively, you can export datasources using the UI:

1. Open Sources -> Databases to export all tables associated to a single or multiple databases. (Tables for one or more tables, Druid Clusters for clusters, Druid Datasources for datasources)
2. Select the items you would like to export.
3. Click Actions -> Export to YAML
4. If you want to import an item that you exported through the UI, you will need to nest it inside its parent element, e.g. a database needs to be nested under databases a table needs to be nested inside a database element.

In order to obtain an exhaustive list of all fields you can import using the YAML import run:

superset export_datasource_schema

As a reminder, you can use the -b flag to include back references.

Importing Datasources from YAML

In order to import datasources from a YAML file(s), run:

superset import_datasources -p <path or filename>

If you supply a path all files ending with yaml or yml will be parsed. You can apply additional flags (e.g. to search the supplied path recursively):

superset import_datasources -p <path> -r

The sync flag -s takes parameters in order to sync the supplied elements with your file. Be careful this can delete the contents of your meta database. Example:

superset import_datasources -p <path / filename> -s columns,metrics

This will sync all metrics and columns for all datasources found in the <path /filename> in the Superset meta database. This means columns and metrics not specified in YAML will be deleted. If you would add tables to columns,metrics those would be synchronised as well.

If you don’t supply the sync flag (-s) importing will only add and update (override) fields. E.g. you can add a verbose_name to the column ds in the table random_time_series from the example datasets by saving the following YAML to file and then running the import_datasources command.

databases:
- database_name: main
  tables:
  - table_name: random_time_series
    columns:
    - column_name: ds
      verbose_name: datetime