If you haven't already, read Part I first.
So, what am I considering for hardware? I'm considering several options, here is my homegrown solution, if I couldn't afford a Filer/NAS device:
Homegrown student storage system:
Dell PowerEdge 2970 Server
Dell PowerVault MD3000i iSCSI SAN
Dell PowerVault DL2000 disk-based backup system
I planned on filling that MD3000i with 15 1TB disks, and when you get to that level of storage, you seriously need to consider how you're going to back it up. Right now, we protect a little over 15TB with our current backup systems on campus, so I'd nearly be doubling the size if I fed this data into our existing system. Doing full backups over the network each week would take far too long, so I'd need to utilize synthetic backups in order to reduce the amount of network traffic and the size of the backup window required. To do that, I looked at the DL2000 system, with two shelves of disk storage attached to it. This is essentially a 2U Dell Server, with Windows and Backup Exec (or CommVault) installed, and shelves of disk attached to the back of the server to use for storing the backups. While we typically use NetBackup, I've fairly familiar with BackupExec, and knew that it could perform synthetics necessary to reduce network traffic. Purchasing a dedicated backup system for this project also made some sense because I could predict the annual cost of the backups quite easily. Our NetBackup system is run on a charge-back basis, based upon how many GB each "customer" on campus is protecting with the system, as that number would fluctuate wildly, so would backup charges. Using our NetBackup system would be cost-effective if the student storage system stayed small, say under 2 TB, but if it hit 10TB, it would be extremely expensive to feed into our other backup boxes.
Now, if I go back to my list of possible protocols that would be useful in this project, which can I provide with this solution?
CIFS - This is easy, it's built into the Windows server I'd use
NFS - Should be possible to do in Windows, maybe with Services For UNIX, but we've really never done it before, and permissions can be VERY tricky. NetApp has a white paper laying out the problems associated with mapping NFS to CIFS permissions, and how they tackle it. I'm pessimistic about our ability to solve the problem nearly as well on our own.
WebDAV - Again, this is theoretically possible in Windows too, but we haven't done it before either. It may work easily, it may be extremely problematic for us.
HTTP/Web Access - I have yet to find a good piece of software for doing this, there are plenty for allowing web users to store files inside databases, I have yet to see a good one for sharing an NTFS file system through a web page.
SFTP - Again, should be possible, if a suitable bolt-on SFTP server can be found, though the ugly permissions monster may attack here too.
So what are the pros & cons of this approach?
Pros:
It's relatively cheap
It's as flexible as I can make a Windows server, if I can find the right software packages and make them work.
It's based on Dell hardware, which we have a lot of experience with.
Cons:
It's not redundant, if a hardware problem strikes the server, it's going to be down for 4 hours while we get replacement parts
Implementing so many simultaneous file protocols would be extremely difficult. I have grave doubts about our ability to make all of that work well, and it would take a very long time to accomplish it.
"Only" scales to 45TB. You can hook three MD3000i boxes in tandem, but that's it. I don't realistically expect to need more in the lifetime of this project, but it is a limitation that's smaller than my other options.
Monthly downtime for Windows patches, as well as patches for various file serving protocols. I know a Filer will have better uptime, just because you don't have to install security patches monthly.
Disaster Recovery - In the event that the system is destroyed, purchasing a new one will take time, and restoring the backups to it could take up to a couple days. No big deal if the problem happens over Christmas break, but if it's the weekend before Finals, it would be bad.
Stay tuned for Part III next week.
