Storage
Up to: Tech WG
so Mastodon takes up space, right? How do we manage that?
Currently we just have all our storage on the same Linode compute instance that we run the mastodon instance from. That should probably change at some point to using S3-compatible object storage.
But for now, what the heck takes up all the space?
What Gets Stored
Database
The database is relatively small since it mostly stores text! For reference, as of August 15 (roughly 8 months after the instance started), it is 13.5GB, with statuses maching up 4.5GB of that.
Media Storage
The instance is currently configured to remove media (images, video, audio) from other instances from the cache after 1 day - after that, the media is redownloaded from the hosting instance when accessed.
Of course, we host all media from our members indefinitely, and actively encourage them to post big images and videos because that fills up their stat counters
Cache
Data from other instances that isn't status and other text data is stored in the system cache: mastodon/live/public/system/cache - this currently makes up most of the storage needs.
It includes
- account data, including avatars and header images, which are stored indefinitely and thus need to be pruned
- media from other instances (images, videos in posts)
- custom emojis from other instances
- link preview cards for embedded links
Managing Storage
Media
Media storage is configured from the Mastodon admin interface - https://neuromatch.social/admin/settings/content_retention
Warning: Do not set the "Content cache retention period" as it will remove posts and bosts from other servers! that includes removing the bookmarks, favorites, and boosts of our members! bad to do!
Cache Pruning
We use tootctl to periodically prune data from the server by running these commands every month. At the moment this keeps our storage in a sustainable range:
# remote cover images from accounts that nobody on the instance is following
tootctl media remove --remove-headers
We might also want to add these commands that make sense to do, but dont' necessarily contribute a ton to our storage burden. Want to check with the rest of Tech WG first before I do these (-Jonny 23-08-15)
# remove remote accounts that no longer exist
# excludes accounts with confirmed activity in the last week in case the server is down
tootctl accounts cull
# remove files that do not belong to media attachments.
tootctl media remove-orphans
Reference
- Helpful info on managing disk usage with tootctl - https://ricard.dev/improving-mastodons-disk-usage/