jcmem

Overview

jcmem provides Java clustered memory via a remote integer key-POJO value server. The server can save its contents to and restore it from a file.

A cluster of clients makes an unlimited amount of memory available to the server. As memory fills up, new clients are started in the cluster and these add to the total memory available to the server.

A client has a priority attached to it. Clients with lower priorities are used first, but the load is balanced across clients with the same priority. Once lower priority memory fills up, higher priority memory is used.

Two types of clients are implemented at present. The first type uses the client's heap (RAM) to make memory available to the server and has a low priority, thus it will be used first. The second type uses a file for storage and has a high priority.

Clients can connect from anywhere but are typically started in some commodity cluster. jcmem does not handle client failures gracefully but relies on its contents being saved regularly enough. If a client does fail, restore from the most recent save. This can be changed but will carry a performance penalty and thus is not a serious consideration at present. The cluster is assumed to be local and of good quality. Likewise, to disconnect a client, save and stop the server, and then restart it again.

Once the server is up and running, and has access to memory via clients, any type of object can be stored on it. Remote clients connect to the server and store an object on it, receiving an index or handle in return. This can then be used later on to retrieve or delete the object.

Objects on the server can also be locked which will prevent them from being changed or deleted. When a client retrieves an object from the server, the server in turn retrieves it from a storage client and then pass it on to the client. To boost performance, especially for large objects, a client can obtain the store from the server and then retrieve the object directly from the storage client.

After some initial fumbling, this now works and outperforms retrieval of objects through the server. It does, however, require some locking mechanism. Objects are locked until the client unlocks them and a direct retrieve is a three step process. First the object reference is retrieved from the server and as part of this the server locks the object. Next the object itself is retrieved directly from the storage client and finally the lock on the object is removed from the server.

As a test I ran a 15G server on a 3G 32 bit laptop, with the clients running remotely on three 64 bit machines. Each of these ran 10 jcmem clients and each client provided 500M to the server, using a variety of heap and file storage and eventually adding up to the 15G total. I used several remote clients to populate the 15G space simultaneously.

Another test, using 10,000 objects, showed that the normal retrieve via the server takes 47 and the direct retrieve takes 37 seconds - a huge saving. In this case the storage client and the object retrieval client both ran on the same machine. Mileage may vary, but will direct retrieve is probably better the larger the objects are.

jcmem only deals with the storage and retrieval of bytes converted into and out of Java objects at present. The next step is to index these objects and then to run queries against the indices. After that the traditional database can be thrown away.

The emphasis is on performance and simplicity.

Use

Download and unzip the file, which contains all the source code and from which javadoc can be generated easily. There are several shell scripts available in the bin directory to test with.

jcmem uses Java RMI, so start it up first via

    rmiregistry

Now move to the bin directory and start a new server

    ./objectServer.sh

Note that the server will dump out statistics every 5 seconds. Next add memory to it. Let us first add heap memory to it, as follows:

    ./heapClient.sh 39000 localhost 500000000

The first parameter is the client port, the next the host and the last the size in bytes to add to the server. Run the script without any arguments to get more help.

The server will now show that it has 500M of storage available. How about making 20G available? Just fire up 40 clients in a cluster, each connecting to the server and making 500M available. Simple, especially if you have 40 machines available and connected via a high speed ethernet. It also helps if the tcp buffer window is large....

What happens if the server runs out of space? It falls over, so let us add some slow but abundant file system space to it to prevent this. File space is used as a type of overflow - the moment the server starts using file space it is time to add more nodes to the cluster and to use those to make more heap available.

A file store is added as follows:

    ./fileClient.sh 39001 localhost 1.tmp 500000000

This adds a file client running on port 39001, connected to the server running on localhost and using a local file named 1.tmp to store data to. The file is created with an initial size and a total capacity of 500M and the server will now show a total of 1G of available storage, made up of 500M from the heap and 500M from the file client we just started.

Note that the provided heap client script is really meant for a 500M sized client and will use 750M of heap space (-Xmx750m). I found 500M sized clients to work well but your mileage may differ and you can easily change the script accordingly.

Next let us run some tests.

    ./testStore.sh localhost 10 2.store

This will test the server running on localhost by creating 10 random objects and eventually dumping the server's contents to a file named 2.store. This is just to see if all is working and should be over within a second, after which 2.store will be deleted. Increase the number of objects to test more extensively, but don't forget to fire up additional heap and file store clients if you want to go really big. Also remember to open some ports on the firewall for the server, which runs on port 29999 by default, and for the clients, which can be started on any port but typically in the 39000 - 40000 range.

Next run the test with 10,000 objects which will require around 300M of memory. Note the difference between retrieving objects from the server and directly from the storage client. I note a 25% performance boost if the server runs on a different machine to the storage and retrieval clients, which both run on the same machine.

What happens next

I hope to stop using a traditional database in favour of jcmem. The next step is to add a JNI client to the heap and file clients, but I'll refrain from that for now. Any takers?

I want to add the ability to maintain indices on the stored objects and then on top of that the ability to run queries and then it is goodbye to the traditional database. I did contemplate adding a prevalent type log facility for jcmem but for now is satisfied with the saving and loading to not be desperate for this, which I consider non-core. Also, the performance penalty of the log is not appealing at all and I am happy with jcmem as is.

Principles

I've looked at terracotta, bigmemory, bigtable, cassandra, ehcache, hazelcast, prevayler, trove4j, store4j, vernon and many others but all did not work for me in some way or another, so eventually I wrote jcmem. The following are principles or guidelines in its development.

Java

jcmem needs to run in a heterogeneous environment. I'm sure the same thing can or have been done very effectively in C but Java is a must for jcmem

RMI and multi-threaded

jcmem uses RMI and is thread safe, as it should be!

Simplicity

In a mission critical environment simplicity is a real asset.

Performance

jcmem must be a lot faster than a RDBMS. Prevayler really impressed me and jcmem is something of a prevalent map of objects running on top of a cluster. Java is known for performance jitter and jcmem provides a way to get around that, without having to purchase a real time JVM - do you know how much those monsters cost? Rather than running the JVM with a gigantic and/or real time heap, run a lot of jcmem clients each making 500M chunks available to the server. I'm not even sure that jcmem will suffer from jitter as a heap storage client basically allocates one huge chunk and thus may sidestep garbage collection related performance jitter, but I don't know for sure.

Clustered, remote storage

Otherwise known as cheap and unlimited storage. jcmem is not bound by the constraints of a single machine and adding memory is easy. As the server fills up, new clients are started (manually) and so the memory available to the server grows. A file storage based client provides an overflow and the server can use that when it fills up rather than fall over.

Dynamically scalable

When jcmem runs low on storage, just add more heap (or file) clients to it.

Unlimited

While it is preferred that all of jcmem should run in memory, it should not fall over when that is depleted, so it needs access to file storage as well.

Focus

jcmem can be fiddled with to add logs, improve file storage performance, make it handle client failures gracefully and so forth. Rather than focus on that, it is assumed that a high-availability cluster is used where client failures will be few and far between. jcmem can save and load and it is assumed this will be done regularly enough to not require a performance draining log mechanism. jcmem only stores and retrieves bytes, lots of them in a clustered environment, and for now that is it.

Download

jcmem can be accessed on its SourceForge page.

Version history

V0.3

There is a bug in V0.2 which will cause data corruption in a multi-client environment. V0.1 used store locks to allow for direct retrieval - a client store is locked until the direct retrieve is complete. This carried a heavy performance penalty and its use was deprecated in V0.1, but it was reinstated in V0.2 but then using object rather than store locks. Object locks will be retained as it also allows clients to lock an object while some exclusive work is done on it, but if direct access is allowed then somehow a store needs to be locked as objects in the store are moved around by the store as objects are deleted. This is fine as long as only the server access those objects since the server is made aware of it, but direct access circumvents this and so a way is needed to either lock the store or somehow notify the direct access pointer of subsequent changes.

This will be addressed in V0.3 and worked correctly in V0.1, but is broken in V0.2. If the performance boost is lost in the process, its use will again be deprecated as in V0.1, and it may be done away with altogether. Ideally, it will be retained due to the performance boost and the reduced load on the server, but with the minimum of fuss in a RMI environment where any additional communication carries a penalty.

V0.2

Numerous changes

• Remove storage client lock mechanism and add object lock mechanism
• Change format of store file to include version, object locks and map index
• Not tested as well as V0.1.....

V0.1

Initial release, used client locks to allow for direct retrieval.

Fin

MG Ferreira, Ferra Asset Management, 2010 http://www.ferra4funds.com.