Setup AWS S3 like Object Storage Server on your own machine

Posted on : 14th Aug 2020

Do you want to deploy an AWS S3 like server on your own machine ? If yes, you need a S3 API compatible Object Storage Server. 

What is an Object Storage Server ?

“A server that does object storage”, said the noob. In simple words, it is an application that helps you store your files/unstructured data as a “distinct unit” calledObject.

What are some best available Object Storage Servers ?

Why not use S3 ?

You should if you want to. I used a custom S3 like server because of a small project, which didn't really require S3’s performance at the moment. But because it might in near future, I wanted to use something that is S3 API compatible. So that when needed, I will just change the config, migrate the data and that is all. *also, I like to maintain my distance from those AWS bills*

What options do I have ?

If you are looking for Object Storage Servers, you might have more available options. But we will be looking at some S3 API compatible Object Storage Servers. Some options are : 

The Good Stuff

In this blog post, we will be using MinIO as our pick. MinIO, as their website summarizes, is a “High Performance,Kubernetes Native,Object Storage”. It is built to support S3 API, and supports a distributed environment natively. Plus, it is open source <3.

Installing MinIO on your machine

  1. Install Docker, and start docker daemon. If you don't already have it installed, read “How to Install Docker”.

  2. docker run -p 9000:9000 -e MINIO_ACCESS_KEY=minioaccess \

                        -e MINIO_SECRET_KEY=miniosecret \

                       -v /mnt/data:/data minio/minio server /data

Once the above command runs, you will have the minio image from docker hub in your machine, and a running container for the same. 

  1. And you are done, MinIO is now running on port 9000 of your machine.

I have installed MinIO, now what ?

Just go ahead, integrate it with your application using boto3 or any S3 wrapper you prefer. 

© 2021, All Rights Reserved · Vipin Joshi