Import Latest File from Google Drive Folder
Import Latest File in Google Drive lets admins and analysts import data from Google Drive to a specified database table. Admins/analysts can setup import schedules to sync with data in Google Drive. This feature is similar to Import from Google Spreadsheet.
Instructions#
Here is a quick 3-step process for querying the latest file in your drive's folder in Holistics. It requires you to import data to a reporting SQL database
1. Set up a data source to allow data imports#
Set up your SQL database to allow write access for a given data source. Please check our docs on how to connect your database to Holistics.
2. Set up files in Google Drive#
Holistics imports the most recently modified file in the specified Google Drive. Supported file types include csv, Google Spreadsheet and Excel (.xlsx).
For scheduled imports, please upload the data file to Google Drive folder before the scheduled time for the import job. The most recent data file added will be picked up by Holistics for importing.
3. Create a new data import#
From the 'Data Import' main menu, click on 'Add new data import'. The new data import page opens up.
Select the source type as "Google Drive Folder". Copy and paste your Google Drive Folder URL into the text box field. Click on the 'Validate Source' button below the text box, to run and validate the Google Drive Folder URL.
If this is the first time the feature is used, you may need to click on the browser's "Allow Popup" button to allow authentication with Google.
4. Validate input and destination table structure#
Once the source is validated and you can see a preview of your data file, specify the destination data source, destination schema name and table name.
The column definitions are also automatically generated from the result of the downloaded file. The data type of each column can be modified as you see fit.
Schedules and sync configurations (optional) can be specified. Import job types such as full, append and incremental will determine how the new data is added to your database table, read more about Import Mode configurations here.
5. Schedule/Execute data import#
Schedules for the current data import job can be modified under the Schedules header on the left side panel. Optionally, data import can be executed manually by clicking on 'Execute' link for that import from the 'Import List'.
6. View data import status from job list#
Once a import has executed, either manually or based on its schedules, a new job corresponds to one execution of that import will be created to track its progress. User can click on the refresh icon next to 'Job History' title from import list page to refresh the job list.
User can also click on 'Logs' link to view the detailed logs of each job execution. If an Import Job fails, an email notification will be sent to the job creator.
Supported File Types#
| File Type | Extension | MIME Type |
|---|---|---|
Google Spreadsheet | .gsheet | application/vnd.google-apps.spreadsheet |
Comma Separated Values | .csv | text/csv |
Microsoft Excel | .xls | application/vnd.ms-excel |
Microsoft Excel | .xlsx | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet |
Note: Sometimes maybe there is a mismatch between a file name and its type, for example, a file named real_csv.csv but its MIME type is plain/text.
To check the real file type, right click on that file and choose View details
A panel will show as below
As you can see, the file type is Comma Separated Values (csv)