Data Source Connectors
Data source connectors (formerly known as extractors) are Keboola components used for importing data from external sources into Keboola.
Types of Data Source Connectors
Section titled “Types of Data Source Connectors”Typically, data source connectors connect to APIs external services, like Facebook, YouTube, Gmail and so on. They can also connect directly to an arbitrary database.
They can be grouped by their primary purpose:
- Database data source connectors: SQL Databases and NoSQL MongoDB and data source connectors from generic Storage services
- Communication, Social Networks and Marketing and Sales data source connectors
- Other data source connectors, such as Geocoding-Augmentation or Weather Forecast
For a definitive list of usable data source connectors, see your project Data Sources section or the public list of components.
Working with Data Source Connectors
Section titled “Working with Data Source Connectors”Even though data source connectors are generally designed for automated and repeated data collection, they can be triggered manually at any time.
- For manual import of ad-hoc data, see Data Import in Storage, or our tutorial on manual data loading.
- Configure a sample data source connector (Google Sheets).
- Configure a database data source connector; other SQL database data source connectors are configured in the exact same way.
As bringing data into Keboola is the main purpose of a data source connectors, go the path of least resistance: Get your data in first, and then convert it to what you want it to look like. To give an example, when connecting to existing information systems, do not modify the data in them. Such data conversion can prove to be difficult and expensive. Extract what you need and shape it in Keboola.
Limitations
Section titled “Limitations”Some data source connectors have limits inherent to their sources. Unfortunately, there is not much we can do about it. For example, the Twitter data source connectors will not let you access the history of a particular tweet account beyond a certain point because of the limitations of Twitter API.
A comprehensive video guide on this subject is available on our YouTube channel.