How Do I Organize My Storage Today

- 9 mins read

Wow, it’s been 2 years since I wrote the last storage-related posts (and that was actually my first blog post ever, learn more)

So, two years have passed, and it’s time to write another one.

Briefly introduction

Despite that I started to focus on privacy and security, the cloud storage is still my main storage medium. But don’t panic just yet, I now have decent amount of local storage as well, and thanks to the magic of moden cryptography, any data stored on the cloud is safe and sound.

3-2-1 principle shall never be forgotten, and I personally obey that most of the times.

Now I will categorize all my file types and give detailed solutions.

Files

Photos & Videos

I said photos & videos, which means those I took by myself and usually have sentimental value on me.

So, as soon as the photos are taken, my phone will automatically back them up onto Google Photos, which I pay a few dollars a year for 100 GB, provided by Google One. Usually I don’t take a lot of photos, so it works well for me. Later when I am back home, I connect my phone to my computer and copy all the files out. The second copy remains on my portable, encrypted SSD.

After organizing all those files, I use rclone to copy those files to OneDrive (personal, not business), that remains as the third copy.

Last but not least, as a catastrophic-proof solution, the last copy will be tarballed, compressed, and encrypted. It will be uploaded to AWS S3, in Glacier Deep Archive.

I know Glacier is extremely cheap on storage but ridiculously expensive on retrieval, but usually I will NEVER need to use that backuip, so as long as the storage fee is cheap, I don’t really care. I mean, 0.99$ for a terabyte for a month, who can say no to it.

So in conclusion, there are four copies, remained in at least 2 countries.

Documents

Those are small files, so I literally store them everywhere.

The first copy will be my working copy, so usually it remains on my computer SSD.

The second copy will usually be uploaded to OneDrive (business), if the document is sensitive then it’s likely that it will be encrypted.

The third copy is likely remains on the telecommunication software. Yes I am serious, it’s very likely that it will be on my Saved Messages on Telegram.

Also, sometimes I’ll just write my stuff using Google Docs, so they remains on my Google Drive.

This is a very janky solution, though the files will be in at least 3 countries. But to be honest, I don’t really have a lot of documents to write.

Source Code

I write a lot of programs & scripts, so this is a big deal.

So currently, nearly all my projects are on GitHub, and GitHub is safe enough for me to trust, but not fully, since it is fundamentally a third-party managed service.

As you might know already, I will always have a local copy containing all my projects, whether it was deprecated, archived or ongoing. Sometimes I will work remotely, so some of them will remain in two different computers in different regions.

Usually in mainland China and Hong Kong SAR, since my main work location is in mainland China and my remote workspace is in Hong Kong. They will be synced using Git.

Both my local and remote workspace will be backed up regularly, so all my source code will also be backed up, that remains as the third copy. In the future, I may consider having a fourth copy located on my encrypted, portable SSD.

The source code will take at most a few gigabytes, plus all the assets, so storage space isn’t on top priority. Safety first, since there are projects that are either not publicly accessible or is written for someone else who doesn’t want to disclose the code to the public.

Cryptographic keys

I’ve written a post about this before, learn more at here.

In short, for day-to-day use, I will just plug in my YubiKey. The keys will be encrypted and stored in the persistent partition (also encrypted) of my Tails USB. So that’s two copies.

For the third copy, I will encrypt the key using OpenSSL (AES-256) and then upload to AWS S3 (Standard), and enable the bucket encryption, located in us-west-2 (Oregon).

The reason why it is in the United States is that I don’t really want to touch anywhere else. There isn’t actually a typical reason for that.

Those are three copies that I can easily retrieve, and I also have a fourth copy. I actually splitted and encoded all my private keys into the form of QR codes then printed out. Of course, the private keys are encrypted beforehand.

Well actually GPG keys have encryption on it by default. You know, the passphrase that you need to enter when you sign/decrypt/authenticate.

I then mailed the printed-paper to one of my friends in Taiwan.

For one thing, as is for now, Taiwan is not politically with the PRC government, so it’s very unlikely that my keys will ever reach the governments’ hands. And for the other, I just have a friend living in Taipei, whom I can trust (although not fully trust, since you should never fully trust anyone, unless in certain circumstances. The keys are encrypted, so no big deal.)

Okay, that’s four copies. The private keys takes only a few kilobytes, so safety is top priority.

Screen Recordings

I record my screen a lot, and there are a lot of recordings.

Well technically not just screen recordings, sometimes I will say something from heart and then record them. But this is another topic which will be counted towards Photos & Videos, so it won’t be mentioned here again.

Usually the screen recordings will be sent to my friends, so this is the first copy. The second copy will remain on Telegram’s Saved Messages. And the third copy will remain on my portable SSD.

Quite often the screen recordings are very unimportant files, so I don’t really care about that.

Backups between Cloud Storage

My personal OneDrive (subscribed to Microsoft 365 Family) will be synced (encrypted) to my archive OneDrive (a dedicated account on my Microsoft E5 subscription), regularly.

My personal Google Drive is also backed up, just like the one above.

I plan to sync my business OneDrive altogether to another E5 subscription I have (in different region, as a geo-redundant backup solution. But I haven’t done that. Will try to achieve that as soon as possible.

Production related

Servers

I have a lot of services running in different servers around the world. I’ve written some scripts that will backup my servers to somewhere else.

For now, the servers will be backed up to one of my backup machine in Helsinki, Finland using Borgbackup.

Borg is very uneasy to use, I think. I might switch to something else later. Before I use Proxmox Backup Server, which I think is pretty good, but Proxmox is too heavy for me.

I admit my solutions for server backup is terrible, and I never simulated the recovery procedure. I might need a huge change as soon as possible.

The server which contains my Vaultwarden (password manager) is backed up in a very old-fashioned way, which I wrote a script to compress, encrypt and send it as an email attachment every 6 hours. And I also set a policy to delete old backups in my mailbox.

I know this is terrible, and there’s literally no deduplication whatsoever, and it wastes the precious email storage (which I have 50 GB total). This will also be modified.

Main Workspace

My main workspace (currently) is a lapotato (laptop but performs literally worse than a potato), which I attach my portable SSD to it as my boot drive, currently with Ubuntu 22.04 LTS on it. It is a Samsung PM981a OEM drive which I got for free. I separated my system partition and my home partition. I backed my home partition up remotely to my remote backup machine (more on that later) regularly.

Upon Ubuntu 24.04 LTS came out, I decided to switch to Debian 12, which I use for production servers for a long time. Canonical really pushes the use of Snap, but I personally hated Snap. It’s not like their concept is bad, it just sucks.

I did installed Debian 12 on one of my other SSDs, (it’s an Intel Optane H10, with 16 GB of non-volatile Optane Memory and 256 GB of QLC NAND) it is encrypted using LUKS and LVM enabled on top of it. I chose btrfs as my filesystem, so thanks to the send function of Btrfs, snapshotting and backing up my drive has never become easier.

Since most of my working files are already backed up, there isn’t actually the necessity to backup the entire drive, since the unique files on it are either cache files or browser data which I can retrieve from other places any time.

Remote Workspace

frank@ap-hkg-1:~$ lsblk
NAME                 MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda                    8:0    0   120G  0 disk 
├─sda1                 8:1    0   487M  0 part /boot
├─sda2                 8:2    0     1K  0 part 
├─sda3                 8:3    0    70G  0 part 
│ └─Frank--vg-root   254:0    0 118.6G  0 lvm  /
└─sda5                 8:5    0  49.5G  0 part 
  ├─Frank--vg-root   254:0    0 118.6G  0 lvm  /
  └─Frank--vg-swap_1 254:1    0   980M  0 lvm  [SWAP]

This is the partition layout of my remote workspace, aka the default layout offered by Debian 12 installation wizard.

I pay IDrive 9.9 USD a year for unlimited photos & videos backup as well as 1 TB of PC & Mac backup. I downloaded and ran their Linux daemon on my remote workspace and it will back my system up every day in UTC 8:00. It’s enough for what I do. As always, basically everything on it is already backed up.

Storage Devices

In the end, why don’t we list all the storage devices I own or use?

Devices that I’m not actively using are not listed.

Device Name Location Capacity Usage
Samsung PM981a local 256 GB Local Workspace
KIOXIA EXCERIA G2 local 1 TB Local Storage
Intel Optane H10 local 256 GB Local Workspace (to be migrated)
4 * Intel Optane M10 local 16 GB * 4 Rescue, Live USB, etc.
KINGSTON DataTraveler 3.0 local 32 GB Temporary Data Container
KIOXIA USB local 64 GB Ventoy (live USB)
Samsung FIT USB local 64 GB Tails, confidential data
Google Drive (personal) Hosted by Google 100 GB (Google One) Photos & Videos, some documents
Microsoft E5 * 2 Hosted by Microsoft, in Japan and the UK 25 TB * 25 + 5 TB * 25 Basically everything
AWS S3 Hosted by AWS, in us-west-2 ? Cold Backup

Outro

There you have it, this is my current backup solution. Comment down below if you think that is either good or terrible :)