Personal Expenses Pt. 1

And so we embark on another project that (hopefully) won't be relegated to the forgotten/abandoned bin. This one though stems from an actual need for me to control my personal cash flows but at the same time, I hope to learn a lot more about Networking, Self-Hosting and Systems Architecture.

Overview

First, a little bit about the inspiration for this project. Everyone should at some point start to keep track of their finances to be a financially responsible adult. As I have finally entered the workforce and have access to tools like a credit card, it becomes harder to keep track of my spending with multiple avenues for money to exit from my sphere of control.

Existing Options and Their Shortfalls

Like many of us, I have asked around amongst my friends and researched online for great apps that will help me in this area. I have also previously been a user of apps like Seedly which has allowed me to sync my transactions directly from my bank account but DBS, my primary bank of choice, has suspended third-party sync.

Great Features

I am currently using Dobin, which is a locally developed application that also has the ability to sync to your bank accounts (including DBS, which I am not sure how they do that). They understand that having multiple accounts will involve some amount of self-transfers between accounts and thus have a handy switch to mark transactions as such and thus they are omitted from cashflow calculations. However, despite there being a handy "Net Cashflow" bar that tells you if you spent more money that month than received, the UI is a bit janky despite the clean look and there is no search function as well to find the transactions that you are looking for.

Digital Wallets

The key issue is that there has been a proliferation of digital wallets and since they are separate from the bank accounts that most people have. These wallets do not provide the robust tools that most banks provide to their customers in terms of exporting their data. For example, PayLah! only allows you to export your transactions as a PDF file as compared to a more usable CSV file. This is also true for credit card statements.

This means that you either have to manually add the transactions to your app, which is ridiculous since the entire thing is an automated system, or be content with just labelling the transaction as a "Transfer" or "Other Income" when you move money in or out of your wallets respectively.

note

Dobin currently doesn't support manual transactions which is also ridiculous but fair given that they want to be a "fully automated platform".

Group Payments

For me the greatest issue are the times when I go out with friends and end up paying first for the meal, following which everyone will proceed to transfer money to me. This means that the budgets that I have set in my application do not show the reality of how much money that I actually spent as the incoming money transferred by the other people enjoying the meal should offset the payment made by the person that paid first. This is also an issue for things like flights where making group bookings is by one person is the norm.

Ideal Product

There will always be some tradeoffs when creating such a personal budgeting application. By having to manually refresh your transactions every time, you might not have a live understanding of your spending as data is not pushed to your app but pulled manually. Since statements are created at the end of the month for most digital wallets, they can only be processed once per month.

The key thing for me personally is that since I mostly make digital transactions as compared to cash transactions, I just want a place for me to see all my transactions for the month and just ensure that I am not blowing my budget at the end of it. I think I can reasonably rely on the internal tools of the various banking apps to have some sort of idea of how much I am spending and how much I have left to spend.

Project Scope

There are going to be multiple moving parts to this project. Some of them are infrastructural and will then be reused for future projects while others are dedicated specifically for this project. The end goal of this project is not just the creation of a personal expenses app but also a local applications infrastructure that I can use to create any other application that I want. I will refrain from connecting it to the internet for security reasons but that will be in the pipeline when my skills are sufficient for me to handle that.

Common Infrastructure

I will need to set up the following as common infrastructure:

1. Local DNS resolver that points *.local.bhargav.io to all the various services that I will have running on my local network.
2. Some sort of Docker environment to run the apps as containers.
3. An Auth server that will handle the creation of users and the managing of JWT tokens.
4. A message queue that will handle all the synchronisation of user changes and any other app specific processing that might need to be done.
5. Some sort of automated deployment pipeline like Github Actions but with the target being my local server rather than some cloud infrastructure.

Tools

Django

Going back to my comfort zone here, but Django and its amazing Pythonic ways allows me to create the scripts that I need and add in all the plugins that I could ever ask for, including data processing pipelines and machine learning models (if I ever decide to in the future).

NextJS

This is another classic Web development Framework that I personally love to use that has routing built in so that you can focus on building web pages and their components. It is also built on top of React which is a great framework for building Web apps.

Caddy

This will end up being the main tool that I will use to reverse proxy my requests as well as ensure HTTPS on the requests. It is a wonderful lightweight and easy to customise tool that handles a lot of the grunt work behind the scenes and has modules that you can load to add extra functionality instantly.

RabbitMQ

This is something new to my repertoire of tools as I have never worked with a message queue before. It will be interesting to figure out how to make it work properly. This might even change in the future if there is a better one that I can use.

Bind9

Inspired by Christian Lempa's video of a DNS server running in Docker, and the need for me to have some way of resolving my local DNS addresses to the right IP address, I will just be following his tutorials to set up the home networking infrastructure. I am not a networking professional but I think it will be interesting to learn how to set up the home network and get the site to work. I am also not sure if this component is fully necessary as if Caddy can just broadcast to the whole network where to find the correct Docker containers, that would be great. For now, I will keep it in though.

Docker

This will be used to host all containers that are running my services as well as the Caddy and DNS instances. It's just a great tool to ensure that all the necessary dependencies for each service are present and do not interfere with other services.

Terraform

This will be used to manage all the DNS records in the Bind9 server.

Github Actions Self-Hosted

In order for me to package and deploy the services that I am creating quickly and efficiently onto my local network, I will need to run Github Actions on a local runner so that it can easily access the correct docker environment.

Additional Considerations

For now at least I don't want this infrastructure to be exposed to the internet as that will involve adding additional security measures that I feel I am not equipped to handle at this moment.

Another thing is that I am not sure if putting all the services in Docker will be the right way to go. Christian uses Proxmox to run virtual machines that host his applications and services, which might make it easier for the DNS service and Caddy will only have to worry about reverse proxying on individual machines running a frontend and backend each rather than across multiple Docker services.

Architecture

A diagram for what I think is the right solution can be found here. It will get updated from time to time as new revisions so it will be helpful for me to learn how things have progressed.

Project Timeline

Given that this is a side project for me and will definitely take some time as I have to do my research, I think it will take 6 months or so to be fully set up. I estimate that it will be about 3 months or so for me to set up the infrastructure, a month for the Authentication app and 2 months for the expenses app.

Parting Words

I look forward to getting feedback on this project from people that have walked down this path before, be it improvements that I can make or better ways of achieving what I want to, please feel free to reach out to me at my email. I look forward to this journey to see what we can create at the end of it.