Python & ETL: A Great Match
According to GitHub’s 2019 State of the Octoverse report, Python is now the second most popular programming language, overtaking the scripting behemoth, Java. Python is used for developing websites, Web applications, and GUI (graphical user interface) for desktops.
It’s popular among data scientists and is a great match with ETL (extract, transform, load) because it makes it easy to simplify complex ETL and data engineering tasks.
Python also is popular because of its robust Standard Library that offers an array of modules and allows users to build custom applications without writing additional code. The syntax rules Python uses place an emphasis on code readability, so it helps keep the code clean and easy to maintain over time.
Python supports a wide range of major platforms and systems (just like StarfishETL) so users can work in the operating system they prefer. Its open source framework and tools let the user optimize to his or her precise needs and cuts down on development time.
Python & StarfishETL
StarfishETL recently added Python to its library of scripting languages which already includes Javascript, C#, and VBScript. Using Python with StarfishETL has some powerful advantages. You can:
- Reference Python’s Standard Library or use your own modules to complete tasks in StarfishETL
- Use global code to reference custom functions or global variables from any stage within your ETL job
- Debug scripts from the syntax highlighting editor, with detailed logging options available
- Connect to Cloud data services or pull data from local sources using the StarfishETL Ray
- Use the built-in “Starfish” library to easily perform ETL specific tasks such as lookups, make execution flow decisions, and more
- Perform common data translations tasks such as string manipulation, parsing, and type conversions
- Create global functions and access them anywhere throughout your ETL job execution. Reuse this code across different tasks and projects
Try Python scripting in your next StarfishETL project! Need help? Check out our KnowledgeBase or contact us.
Start a Project with us
Fill out the form below and we will contact you
Danine Midura
Director of MarketingAs Director of Marketing for Technology Advisors Inc., Danine spearheads TAI events, campaigns, SEO and website management, webinars, user groups, and social media efforts. Her goal is to support TAI's mission to listen, personalize, and stay with its customers by crafting honest, helpful, and insightful marketing content. Danine's interests include blockbuster disaster movies, tank tops in an array of colors, used book stores, Jurassic Park, and being surrounded by trees.
