Dublin Bikes: an architecture case study in building under real resource constraints
Dublin Bikes: an architecture case study in building under real resource constraints
Last semester, two classmates and I built a smart bike-sharing app for Dublin residents and visitors from scratch — pulling data from the JCDecaux station API and OpenWeather, with real-time dock availability, a 30-minute availability prediction model, and a three-leg (walk-cycle-walk) route planner.
My part of the project was data infrastructure and deployment architecture: database design, the automated data pipeline and containerised scrapers, system deployment (AWS EC2, Docker Compose, Nginx, SSL), backend integration, and unit/smoke testing across three core modules. A few decisions from that work still stick with me:
1. Dual-database strategy
The EC2 instance our course recommended had 1GB of RAM — nowhere near enough to hold two months of historical data, and mixing live queries with ML training data in one table was already slowing responses down. I split it into a 7-day rolling "Active DB" for live queries and a full 2-month "Archive DB" for offline training, decoupling the two completely.
2. Container orchestration under a hard memory ceiling
That same 1GB had to run the Flask backend (with an ML model loaded in memory), two scraper containers, and Nginx simultaneously. Initial allocation was 0.4 CPU / 200MB per scraper and 0.6 CPU / 500MB for the backend — but once the model was loaded, the backend kept hitting its limit. I brought the scrapers down to 0.1 CPU each (they're I/O-bound, not CPU-bound) and raised the backend to 0.8 CPU / 700MB. Gunicorn also moved from multiple workers to a single worker with 4 threads, since each extra worker would have meant loading another full copy of the model into memory.
3. Backend integration: my first real project in Flask
The course recommended Flask. With three of us building independently, the seams showed fast — inconsistent response formats, each module opening its own DB connection, and a couple of endpoints calling the external API directly instead of reading from the database. During integration I pulled this into one structure: Blueprints splitting routes by feature area, a shared response wrapper, a single pooled DB connection, and one hard rule — live data only ever comes from the database, written by the scrapers, never fetched ad hoc.
I'd built a few Spring Boot projects before, but this was my first time doing the full stack in Python, and not from a tutorial — it was three of us figuring the architecture out as we went.
Afterwards I added a 5-minute in-process cache via Flask-Caching rather than Redis: 1GB couldn't afford another running service, and on a single-instance setup, Redis would have been over-engineering anyway.
4. Data quality and test coverage
The two scrapers had to run unattended for two months to build the training set. I added request timeouts, transaction rollbacks, forced connection pool cleanup, and a drift-compensation routine — without correcting for small network delays, two months of continuous polling would have introduced systematic timestamp drift into the time-series features feeding the model.
Alongside that, a full pytest suite (unit + smoke tests) mocks every external dependency — database, third-party APIs, the ML model — for 92% coverage.
The project originally ran on AWS EC2. After the course ended, I recently moved the whole system — app and database — to an 8GB Hetzner server I already use, partly to cut cloud costs, partly to give it a home where it can keep running instead of sitting idle after submission. The move surfaced a few things I hadn't dug into properly the first time round:
5. Splitting database credentials by least privilege
On RDS, one account had done double duty as both schema admin and daily application user. This time I split it into two: a root account used only once, for initialisation, and a scoped application account with access to nothing beyond its own databases. It's one thing to know "least privilege" as a principle — actually doing it makes clear how much it narrows the blast radius when something goes wrong.
6. From dedicated ports to a shared reverse proxy
The original plan was straightforward: deploy the frontend the same way as before, with its own Nginx handling SSL on ports 80/443. It didn't work — this server already runs another project of mine, and that project's Nginx container had those ports locked down first. Two apps can't bind the same port on the same host.
I looked at three ways to fix this: give each project its own Nginx and manually write reverse-proxy rules between them, use Nginx Proxy Manager for a GUI-based setup, or use Traefik. I went with Traefik, mainly because of where I expect this server to be in six months — more projects, not fewer. With Traefik, adding a new project means dropping a handful of Docker labels into that project's own compose file; Traefik picks them up automatically and starts routing to it. Nginx Proxy Manager would have meant going back into a separate admin panel for every new project, and the routing config would live outside any repository — not something I could track alongside the code it belongs to.
So all incoming traffic now hits a single Traefik container on 80/443, which reads the hostname on each request and forwards it internally to the right project's container — Dublin Bikes, or whatever else is running alongside it. TLS certificates are requested and renewed automatically per domain. (One of those Docker labels had a colon where it needed an equals sign, which cost me a fairly long evening with the logs before I found it.)
Last semester was mostly about making it work — and making it hold up inside a 1GB memory ceiling without falling over. Coming back to migrate and maintain it independently, the questions were different: how do I make this safer, and how do I make it something that keeps running long after I've stopped actively working on it. That shift is probably the most concrete way I can point to how my sense of "engineering" has changed this year.
Live demo: bikes.yuhanwang.dev