com.purplehillsbooks.json

Class LockableJSONFile

java.lang.Object

com.purplehillsbooks.json.LockableJSONFile

public class LockableJSONFile
extends java.lang.Object

For use when you have a file being shared across a cluster of servers in order to assure that only one node of the cluster has access to the file at a time.

This is a replacement for another class ClusterJSONFile which worked for locking the file but did NOT allow threads to lock each other out of the file. This approach is different in that you have one shared object that represents the file, and you can use a synchronized critical section to keep threads from stepping on each other. The underlying lock protocol is the same and so the classes are compatible on the operational side.

The object that represents the file allows you to lock a proxy file it before you read, replace or rename the protected file, and then unlock after you write, offering full file lock concurrency control across read and update. The standard file lock does not work because the JSONObject writes to a temporary file before the old file is deleted. Only when the temporary file is fully committed to the disk, the old file deleted, and the temporary file is renamed.

Instead, it locks a symbolic name, based on the original file passed in.

 Actual JSON file:     c:/a/b/c/file.json
 Symbol is locked:     c:/a/b/c/file.json#LOCK
 

While the file is locked by one node, the other nodes trying to read the file are blocked. When one node unlocks the file (after writing) then any other waiting node is allowed to proceed.

This clearly only works when all the code uses the same approach. Use this class in a cluster and it will prevent one node from writing over the changes of another node. It is kind of heavy handed. Clearly if you are doing high performance you should use a database, but this approach is suitable when you have files that are read a modest amount and updated infrequently.

USAGE - update existing file

 File myFile = new File("c:/a/b/c/file.json");
 LockableJSONFile ljf = LockableJSONFile.getSurrogate(myFile);   //retrieve
 synchronized (ljf) {
     try {
         ljf.lock();
         JSONObject jo = ljf.readTarget();       //read
         ...                                     //exclusive actions while locked
         ljf.writeTarget(jo);                    //write
     }
     finally {
         ljf.unlock();
     }
 }
 

Get the lockable file just before the use. In order to prevent overlapped reads and writes do the file operations within a synchronized block of Java code, using the lockable file as the synchronization object.

The expectation is that the underlying JSON write mechanism will write to a new file, and then delete the original file and rename the new file to the real name. Because of this approach you are generally safe reading a file without a lock if you are NOT going to update the file. But if you plan to write to the file, you must get the lock before you read the file.

USAGE - creating a file

 File myFile = new File("c:/a/b/c/file.json");
 LockableJSONFile ljf = LockableJSONFile.getSurrogate(myFile);   //declare
 JSONObject jo = new JSONObject();                  //create JSON contents
 synchronized(ljf) {
     try {
         ljf.lock();
         ljf.writeTarget(jo);                       //create the file
     }
     finally {
         ljf.unlock();
     }
 }
 

The file is not locked guaranteed to be locked after the call, but if an exception is thrown in the middle, it might be in a locked state, so be sure to call unlock in a finally block. If you want to read and update the new file, use the regular readAndLock method to assure that this process has exclusive access, just like normal.

USAGE - reading but NOT updating a file

 File myFile = new File("c:/a/b/c/file.json");
 LockableJSONFile ljf = LockableJSONFile.getSurrogate(myFile);   //declare
 JSONObject jo = ljf.lockReadUnlock();
 

This convenience method does all the required locking and unlocking of the file in a single method. You may NOT update the file after reading it this way without implementing the full pattern for update.

USAGE - exceptions

 File myFile = new File("c:/a/b/c/file.json");
 LockableJSONFile ljf = LockableJSONFile.getSurrogate(myFile);   //declare
 synchronized (ljf) {
     try {
         ljf.lock();
         JSONObject jo = ljf.readTarget();              //read and lock
         ...                                            //exclusive actions while locked
         ...                                            //after all this is done, only then:
         ljf.writeTarget(jo);                       v   //write and release lock
     }
     catch (Exception e) {
         throw new Exception("Unable to ... (details about goals of this method)", e);
     }
     finally {
         ljf.unlock();                                  //unlock WITHOUT writing content
     }
 }
 

It is important to assure that if you lock the file, you also unlock it. If you have multi-threading, then do everything in a synchronized block. Exceptions can always occur, so do everything in a try block, and put the unlock in a finally block. You must lock before you do anything, even just checking existence. It is OK to call unlock redundant times, the unnecessary unlocks are ignored, however following the pattern of the finally block virtually eliminates this possibility. If you hit an error, you generally don't want to write out the file with an undetermined amount of update to the content. Unless you know how to 'fix' the problem, you generally want to leave the file untouched, but you want to clear the file lock so that other threads have access. The exception might not be the fault of the file itself, and it may not be a problem next time you read it.

RQ High Reliability Option

We have a problem at a customer where they are using a very unreliable file system. This is causing errors. The problem is that when a file is written, and unlocked by one system, the other system gets the lock, but is unable to read the file. Sometimes because the file is locked and unable to be accessed. Other times the temp file can not be renamed for some reason.

The strategy to avoid problem is:

1. When getting the locked file, also check that the target file exist. Wait for it in increments of 20ms for up to 1 second for it to appear.
2. When reading the file, wait for the file to exist before reading it.
3. If there is a failure read the file, wait 50ms and try again, up to 5 times.
4. When writing a file, it there is a failure, wait 50ms and try again, up to 5 times.
5. After writing, wait for the file to exist.

Method Summary

Modifier and Type	Method and Description
boolean	exists() Test to see if the target file exists.
static LockableJSONFile	getSurrogate(java.io.File targetFile) Get a lock file surrogate object, that is an object that represents the file being locked / read / written.
boolean	isLocked() Tells whether the calling program/thead is holding the lock.
void	lock() This is the basic lock command and wait until the target file is there.
JSONObject	lockReadUnlock() The easiest way to safely read a file.
JSONObject	readTarget() Read and return the contents of the file.
JSONObject	readTargetIfExists() Read and return the contents of the file if it exists.
void	unlock() Use this to unlock the file when you don't need to update the contents.
void	writeTarget(JSONObject newContent) This will update the contents of the file on disk, without changing the lock state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getSurrogate

public static LockableJSONFile getSurrogate(java.io.File targetFile)
                                     throws java.lang.Exception

Get a lock file surrogate object, that is an object that represents the file being locked / read / written.

Throws:

java.lang.Exception

lock

public void lock()
          throws java.lang.Exception

This is the basic lock command and wait until the target file is there. It will block until the lock on the LOCK file can be gotten. Then it will check if the target file exists, and wait up to 1 second for it to appear. You should only call lock once, before calling unlock. It waits for the file to appear because on some shared file systems, the lock signal will arrive long before the file name change from the last update will propogate. That means that for a long period of time (sometimes more than 100ms) it will appear like there is no target file in the file system. To prevent errors we wait for it here. After 1 second it will continue without error assuming that this is the extremely rare initialization case. That means the very first time you create a file you will encounter a 1 second delay but we assume that is a rare case that is not seen in normal operation.

Throws:

java.lang.Exception

unlock

public void unlock()
            throws java.lang.Exception

Use this to unlock the file when you don't need to update the contents. This method is particularly useful in 'finally' statements, where an error has occurred, and you simply need to make sure that the lock is released, while leaving the file unchanged.

Throws:

java.lang.Exception

isLocked

public boolean isLocked()

Tells whether the calling program/thead is holding the lock. It does not tell you whether any other thread or program is holding the lock at the current moment.

exists

public boolean exists()
               throws java.lang.Exception

Test to see if the target file exists. Generally, you need to assure that the file exists, and to initialize with an empty JSON structure if it does not exist. So use this to test whether you need to call initializeFile.

Note: since the smallest JSON file has two characters (just the open and close brace) this method will return FALSE when an empty file exists at that name. The file must be 2 byte or longer to be existing according to this routine.

Note2: file must be locked BEFORE calling this to be sure that it does not change in the mean time.

Throws:

java.lang.Exception

writeTarget

public void writeTarget(JSONObject newContent)
                 throws java.lang.Exception

This will update the contents of the file on disk, without changing the lock state. The file must be locked before calling this.

Throws:

java.lang.Exception

readTarget

public JSONObject readTarget()
                      throws java.lang.Exception

Read and return the contents of the file. You must lock the file before calling this.

Throws:

java.lang.Exception

readTargetIfExists

public JSONObject readTargetIfExists()
                              throws java.lang.Exception

Read and return the contents of the file if it exists. If it does not exist, the file will be created with an empty JS object. You must lock the file before calling this. Note: this method will first wait up to 1 second to see if the file exists, and then it will initialize the file to an empty object if not found, so that next time it will see and read the file quickly. This method should only be used on files that almost always exist and should be initialized if they don't exist.

Throws:

java.lang.Exception

lockReadUnlock

public JSONObject lockReadUnlock()
                          throws java.lang.Exception

The easiest way to safely read a file. Use this to read the file if you are NOT going to update it. It locks the file briefly, reads it, and guarantees that the file is unlocked at the end. It is synchronized so you don't need to do a synchronize on the object. This is the simplest way to safely read a shared file.

Throws:

java.lang.Exception