Dump and backup a database on shutdown
I’m using Multipass as the virtualisation tool for quickly setting up virtual development machines on my Mac Studio using cloud-init for configuration and setting everything up. This really works great and has saved me several times where stuff crashed and burned, it was really easy just to tear everything down and re-run the setup scripts. (You can read more on my setup in the repository I use for this. This works fine as my development stuff is mostly in stored in Git and the data in a shared MySQL virtual server but as I recently found out this is not lways the case. Sometimes there is local data on the virtual server that you would like to keep.
The solution I came up with to prevent the loss of data is to trigger a script on the shutdown of the server that would copy the relevant data to a safe location. In my case that would be an S3 bucket. I took some digging, searching and testing but I got it working. So if you are looking for something similar, here how I did it:
We use a system service that runs at the start of the shutdown proces, so that other services that we rely on are still running. I’ve named it my S3shutdown.service which is the name of a file which you need to create in
/etc/systemd/system/ with the follwing content:
[Unit] Description=Save database to S3 Before=shutdown.target reboot.target halt.target [Service] Type=oneshot RemainAfterExit=true ExecStop=/home/ubuntu/projects/dumpandstore.sh [Install] WantedBy=multi-user.target
Where the first line is a descriptive title which you will see used in syslog when it is executed. The last line defines the runtime, so before the multi user mode ends. Referenced by
ExecStop you reference the shell script that should be run at the moment the server is going down.
dumpandstore.sh script looks like:
#! /bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin /usr/bin/mysqldump -uuser -ppassword databasename > /home/ubuntu/projects/databasedump.sql; today=$(date +%Y%m%d); cp /home/ubuntu/projects/databasedump.sql.sql /home/ubuntu/projects//databasedump$today.sql /usr/bin/gzip /home/ubuntu/projects//databasedump.sql$today.sql /usr/local/bin/aws s3 cp /home/ubuntu/projects/databasedump.sql$today.sql.gz s3://mybucketname/ /usr/local/bin/aws s3 cp /home/ubuntu/projects/databasedump.sql s3://mybucketname/
I’ve used a dump with a data to build some historic perspective, the other file without data is so to speak the last copy and is also referenced in the build script of the server. So that when I rebuild the server the database is filled with the last used dataset.
To activate the service you’ll need to run the command:
sudo systemctl enable S3shutdown.service Reboot the machine and everything should be working as intended. Some problem struggled with was the aws comfiguration. I had setup the aws configuration including credentials as a normal user but the shutdown service runs as root and therefore the aws command cloud not locate the proper credentials. This was quicky solved by copying the
~/.aws directory to
/root Not ideal but it made it work for the moment, I need to do more research for a more elegant and safer solution.
The command line is the future of interaction
Yesterday I read this blog post from Lukas Mathis. It kept resonating with me as I recognised some of the scenarios and could even come up with some more. As an avid Apple user and forced to work on a Windows based platform I regularly need to google for instructions on how to perform certain tasks. I want to make a special mention to the Microsoft Office suite where the menu structure is unintuitive with an inconsistent toolbar where I also end up using the Help function a lot to find the right option.
This could mean a big change in UI design moving away from WIMP, touch interfacing and could be the precursor to voice interaction with the desktop. This is already possible especially on a Mac with Accessibility options enabled. But for that to happen in a common setting we first need to enable sub vocal interaction. It is socially awkward to talk to your mobile or computer in a public space and can you visualise an office space with everybody talking to their computer, very noisy. I recently saw this post which means that it won’t take long before we see a practical solution to this.
Looking forward to this…
The DRI of your career #
I found this a good read, it set the focus on your career instead of your current job. (DRI meaning the Direct Responsible Individual.) It also talks about the all important work/life balance and the focus on growth. I like the idea that your current job is not the end goal but a stepping stone into your career. Look at what your current job can bring in value to you and your career, if the balance ends up to be negative then find the next job.
At DuckDuckGo, there’s an expression: “You are the DRI of your career” (DRI: Directly Responsible Individual). I like this, both as an individual who has always felt like the DRI of my own career, and I like it as a manager because I think it makes the boundaries of what you can and can’t do for people clear.
It reminded me of the time I was a manager and scared one of my team members by suggesting that if he wasn’t happy I would happily help him to find an opportunity elsewhere.