SpaceX Launches
This project tracks and presents an overview of past and upcoming SpaceX launches.
The data pipeline is structured as follows:
Currently the pipeline runs once a week, is triggered by a cron job, and uses Github Actions as an orchestrator.
Why build this?
Space is cool. Data is cool.Combining space and data is even cooler.
As an added bonus, it was also a good opportunity to try out tools I hadn't used before.
Aren't there many SpaceX dashboards on GitHub already?
Yes, but a lot them use an outdated API.
Additionally, I believe effective dashboarding prioritizes the end user. Many SpaceX dashboards seem to visualise every data point possible.
In contrast, my dashboard focuses on presenting key essentials (launch count overview, past and upcoming launches).
Why only SpaceX?
No particular reason. I might add other space agencies in the future.Why use **insert tool choice**?
dlt - For a simple task like this, it didn't make sense to use other heavy ingestion tools. Just using python would've also worked but I was curious to try out dlt.
dbt - Gold standard for data transformation , offers built in data quality checks, and works really well with Motherduck unlike sqlmesh (which works but is not as intuitive).
Motherduck - This one's debatable , honestly the main reason is that their free plan is quite generous (unlike Snowflake that has an expiry date).
evidence - Looked really nice , unlike other BI tools their offering works really well for a static site aka personal projects. I also liked their code first approach rather than the usual drag and drop interface with other tools.