As part of our #whiteboardseries, we bring to you a series of short videos that we hope will help your organisation increase productivity and efficiency.
In the video above, Mr. Sunil Uttam, Co-founder, Mithi Software, we explain various methods available to migrate large volumes of email data from the customer environment into the archival platform, including demonstrating a live migration using AWS Snowball.
To migrate mail from active or inactive mailboxes we suggest using the IMAP sync. Whereas to migrate PST and EML files you can use either the internet (for small volumes) or the Amazon Snowball (for large volumes).
Good afternoon friends, welcome to the next episode of our whiteboard series. I am Sunil Uttam, Co-founder and Principal Solution Architect at Mithi. My primary role is to help customers choose the right solution and design the right architecture for their needs of data management and business email.
So in this video we are talking about, a very hot topic which goes on in our discussions and that is on how do we migrate large volumes of data from the customer environment into the archival platform. While it is easy to get started with archival this becomes a challenge. In this video we’ll try to clear out the various methods which are used. Let’s get started.
So there we have your environment, which covers the two basic areas where you store data. One is the live environment which is your cloud email solution or a mail server in your environment, where you are running live mailboxes. Or there could be mailboxes of inactive users. So that is one part of your environment.
The second part of the environment is over the years you would have been collecting and storing PST files and EML files which you get from client PCs and local archives from PCs and laptops which you would be storing and backing up on possibly USB drives or other network drives. So these are your two points where your data really originates, your historical data.
Import Data from active or inactive mailboxes
Now let’s look at the first one, so if there is a need to move data from the current mail boxes, inactive or active into their respective vaults on Vaultastic, what you might want to do is, instead of exporting the data into PST and EML and then converting it and transporting it, there is a way to do it directly, where we do an IMAP to IMAP sync.
So we would run a job on Vaultastic which will pull those servers, respective users on each of those servers and copy the data using the IMAP protocol, across the internet. Securely though, not to worry, into the respective vaults. So the reason you would use this, is to recap that you do not want to get the data out of those systems at step 1 and then copy them at step 2. You can just do a direct copy. Now you would use the internet and this would be one of your bill of materials for this action.
Now, let’s look at this one, where you have got these maybe terabytes and terabytes of EML and PST lying around and you need to make sure they are safe and secure on the cloud. So what you want to do is, copy these and convert them into the vault format.
Importing data from PST and EML files
One is, using the internet, this is useful if the volume is low, not too much and you could copy on your last mile, because there is a last mile involved here which will be typically from your environment to the Amazon cloud. So we want to make sure that, this volume is not so much that it takes you months and months.
So if it is a small volume, internet is a preferred means, where what you would do is, copy these files, onto an S3 bucket which is in your Amazon account. Which is your hold or your storage and then using a convert tool which Mithi provides you from the Vaultastic application, you convert this data from PST, EML and mbox formats into the vaults. And this is an automated procedure, you can define jobs and just monitor them, that should do.
So this is one way, but if the data is very large, if it is in Terrabytes, 10s, 20s, 40s, then what you want to do is, you want to physically copy this data to Amazon cloud. So there is a technique for that, or there is a tool for that, called the AWS Snowball. It is a physical device, which will be shipped to you, in your environment, you copy the data there, ship it back to the Amazon cloud. This will be connected to your VPC or the virtual private network which you own and you can copy this data into the S3 bucket and then run the same procedure which I described
So these are the two ways in which this large volume of data can be moved into the cloud.
Why you would use snowball is when your data size is very large and moving it on the internet will take you eons right. So that’s the reason you want to do it.
Live Migration Using AWS Snowball
Now, as I am speaking to you, there is a live migration going on, using the Amazon snowball, for few of our customers, and I plan to actually show this to you from our data centre. Right, so here we are in the Mithi Data Centre, which is in out office and I thought, I’ll just show you live migration happening while, we were shooting that video.
So this is an Amazon Snowball device which we have ordered, which comes to us by courier and it comes, just the way it is. There is no packaging to it, this is the packaging, and a very robust packaging for it. This device connects to the network, and there is an amazon snowball client which we install on our PC using which, we can copy files onto this.
So right now, file copies are going on. We are migrating the data of about 10 Terrabytes for few of our customers to do it in one stroke. This device will be shut down, when we are done copying, all the data will be encrypted automatically and this will be shipped back to the the Amazon DC , via courier, where it will be connected to our VPC and from which we would copy the data back into the Vaultastic platform and convert it.
So this is one of the mechanisms which I described to you of transporting large volumes of data from your setup to the Amazon DC. Right, so I hope that was nice, you had a look at the snowball in action.
Now, just to recap and conclude, what we have described is, you have mailboxes, active, inactive from which you want to transport data to Vaultastic. Alternatively, you have PST, EML and mbox files in large volumes on your environment which you need to migrate.
For live mailboxes, we suggest the IMAP sync, although it will take a little while, it is fully automated and you don’t have to worry. You have to only pay for the internet charges which you bear on the last mile. and for these, you have two ways.
For smaller volumes, choose the internet, which again goes onto your last mile and for larger volumes choose amazon snowball which again goes onto your account. You put it onto your S3 bucket, which is in your Amazon account, it is in your control, you own this, and just allow us access or allow Vaultastic access to do the conversion for you.
Want an in-depth discussion?
Our archival experts are ready to talk to you, one on one to suggest archival best practices, to help you manage email storage better, learn how to make use of all the archived business email for business intelligence and stay ready for regulation compliance. Contact us Today.