RIndMA Disk

Some server and workstation applications are hampered not by insufficient processor speed, but storage latency.

For several years already database hardware is backed up by solid-state RAM disks. There are several solutions on the market that (more or less) fit to most database users. But when I tried to implement the existing solutions for workstation acceleration, I found out that they just do not fit for the task. The cheap solutions can speed up random database access, but when working with big scratch files, they are inferior to the best of the common hard drives. The high-end solutions do much better, but the price is killing. That seemed to be the dead end, by I fount the solution. It is so simple the may not be patented in most European countries and might be implemented by any scholar who can hold a screwdriver and know something about any flavor of UNIX. And it outperform cheap solutions both in productivity and price (so, might be used in the low database servers market and middle and high-end workstations market) and even may compete with high-end solutions on the small and medium segment of database servers market (woud not fit for Coogle use, but do you need Coogle solution?).

This article describes the second version of the RIndMA Disk - with Infiniband connection faster than previous 10GB Ethernet version, though a bit more expencive (well, really, with componets price drop even a bit cheaper than initial version)

(Remote Indirect Memory Access Disk)

- Server class motherboard
- 1-4 server processors (depends on processor architecture)
- 32-128GB standard DDR2 server-grade memory
- System disk and backup disk array
- 2 x 20-40Gb Infiniband links
- Case, PSU, keyboard, mouse, cheapest monitor
- UPS
That was the hardware.

Any flavor of Unix (well, even Windows XP or Vista), RAM disk software, Infiniband, IPoverInfiniband drivers and InfinibandOpenSM manager, scheduled backup software, power failure triggered backup software

That's all! Just connect the external RAM-disk to your system and use it.

Price (Open Solaris, with data backup, w/o UPC ): ~$4200 for 30GB solution, $6000 for 62 GB solution, $10500 for 124 GB solution

Reported throughput of 2 x 10GB Ethernet version is around 1.2-1.5 GB/sec, no data for Infiniband one yet

Separate UPS is recommended for database server RIndMA Disk, but may share the workstation UPS.

May also serve as a workgroup archive file-server (with larger [and/or more] HDDs)

Sample RIndMA Disks

The rivals - low end

Not so many RAM disk solutions exist, so I can give short description for most of them.

Note: Useful for workstation solution RAM disk must be around 40-60 GB, so I'll compare 30-32 and 60-64 GB solutions (where applicable)

Gigabyte i-RAM BOX

The cheapest existing solution.

Single i-RAM BOX may have up to 4GB of memory (up to 8GB with special driver) on board and uses SATA/150 interface. Has an optional power-failier battery support. Price for 8 GB solution is around $600.

32Gb solution (5 x i-RAM BOXes at RAID 5) will cost (including outer box, PSU and SAS RAID adapter, no batteries) around $3700 and will allow about 500 GB/sec throughput.

For 64 GB solution cost is doubled, but the throughput will not increase, as there will be two logical drives that might not be used by one program simultaneously (well, seems it will not fit for database server at all).

(With ACARD ANS-9010 out the device becomes absolutly obsolete)

ACARD ANS-9010

The only solution that at price/performance may rival with RIndMA Disk. A single ANS-9010 5.25" drive supports up to 8 memory modules (up to 64 GB total capacity), has 2 SATA II interfaces, battery support and built-in CF slot for automatic data backup / restore (the vendor states that battery capacity is enough to automatically backup data at power fail).

The unit price (w/o memory and backup CF card) is about $400. A cheaper (with one SATA II port and 6 memory slots) version - ACARD ANS-9010B - costs about $250. The unit drawback is that its SATA port is too slow - less than 200MB/sec with one port and no more than 300MB/sec with two ports connected (so the interface is really SATA I+).

About 10-20% slower than RIndMA Disk, ANS-9010 64GB 4-units (8 ports) RAID 0 array (each drive with 8GB memory and 8GB CF card) will cost about $4000-4400 (w/o case and RAID adapter).
ANS-9010 was annonced about two months later than the first version of RIndMA Disk description was published (and became available only quite recently)

Curtis Nitro.XE and HyperXCLR

Curtis has two low-end solutions. I would not describe in full Nitro.XE - this 12GB drive uses outdated SCSI interface, has only 80 GB/sec throughput and is priced at about $2650. Has built-in power-failier battery support

HyperXCLR is much faster (uses Fibre SCA 40-pin connection) - up to 200 MB/sec and may have 18GB of memory on board. Costs about $2250. Has built-in power-failier battery support

36 GB solution will work at up to 400 MB/sec and will cost (including 2-port adapter) around $5000-5500

58 GB solution will work at up to 800 MB/sec and will cost (including 4-port adapter and case) around $11500-12500

Curtis also has rack mount solutions (up to 40GB).

(With ACARD ANS-9010 out the device becomes obsolete)

Attorn/HyperOS Hyperdrive

Single Hyperdrive may have up to 32GB of memory, but utilizes narrow SATA/150 and will cost about $11 000. Has an optional power-failier battery support.

64GB solution is two Hyperdrives in RAID 0 (with double throughput). The price of 64 GB solution (no batteries) will be $22 000.

32-64GB Hyperdrive solutions usually do not need extra box or adapter.

(With ACARD ANS-9010 out the device becomes obsolete)

There also exist Vmetro NVRAM Cards (up to 16 GB PCI cards), but to install more than one there is a need for external PCIe-to-PCI expansion and I just could not find them at the stores and there is no any pricing on them.

The rivals - high end

All high-end solutions have data backup to HDD installed

Violin Memory Violin 1010

Min. 120 GB at starter kit 2U rackmount case with x4 PCI Express interface (up to 2 GB/sec) at just a bit less than $50000 (yes, 50K, and it's the cheapest one!). Needs external UPS.

May be expanded to 504GB and may be supplied with x8 PCI Express interface

Solid Access USSD 200

2-channel SCSI (320 MB/sec), SAS (600 MB/sec) or Fibre-channel(800 MB/sec) 2U box. Can contain from 16 to 256 GB of memory. Needs external UPS

32 GB box - $28000, 64GB box - $49000, 128 GB box - $75000

Texas Memory Systems RamSan-400

3U box supplied with 32-128 GB on board. 2-8 Fibre-channels or 1-4 InfiniBand connections (up to 3000 MB/sec). Has 3 built-in independent UPS's
Price for 128GB stuffed - $90,000 (no other pricing info)

There are also lighter (RamSan-300 - 16-32 GB - $20000 for 16GB) and heavier (RamSan-440 - 256-512 GB - $150 000 for 256GB) versions

PS.

This article on RIndMA/RDMA drive was written a couple of months before Intel annonced their new SSD and ACARD presented their ANS-9010.

At price/performance level ANS-9010 is the only RAM solution that may compete with RIndMA Disk and Intel X25-M and X25-E, but at the productivity level the situation is a bit different. In SQL server model RAM-SSD are still by far outperform flash-SSD (even best HDD are better for the market), but with bigger write blocks Intel SSD are about 25-30% faster than ANS-9010 (and, with 8 disks in RAID 0 array, even a bit faster than 2-ports RIndMA Disk)

New 200+ GB/s flash SSD of other vendors with much lower prices (less than $200 for 30-32 GB unit) due in the beginning of 2009 will make the competition even tougher.

Quite possible than in 2-3 years RAM SSD will become absolutly obsolete.

PPS.

With new Intel 5500 and soon expected 7500 series processors the ServerRAMdisk (as well as RAM-SSD) is of no sence for workstation use. Soon-to-come Supermicro X8DTG-QF motherboard is nearly ideal workstation solution. Only ultra-huge projects will need more temp/scratch/buffer memory.