com.purplehillsbooks.streams

Class MemFile

java.lang.Object

com.purplehillsbooks.streams.MemFile

public class MemFile
extends java.lang.Object

Holds a stream of bytes in memory. It is a buffer that you can stream to, and stream from, in either bytes or characters. More efficient than a byte array since the bytes are not held in a contiguous array, and the bytes do not need to be copied around in order to keep the byte array contiguous.

To write bytes to the memory file, either 1) Get an output stream and write output to it 2) Instruct the memory file to fill itself from an InputStream

To read bytes from the memory file, either 3) Get an InputStream and read from it 4) Instruct the memory file to write itself to an OutputStream.

For character-oriented reading & writing, only UTF-8 character encoding is supported, because that is the only encoding that can represent the entire Unicode set without loss.

For getting characters into a mem file you can: 5) Get a Writer to write characters to the memory file 6) Instruct to read all chars from a Reader into the mem file.

for getting characters out of a mem file, you can: 7) Get a Reader to read characters from the memory file 8) Instruct to write all chars to a Writer.

Usage and Justification: The main usage is that you need to construct the contents of a file programmatically, and then parse it, or alternately you need to write something out, and the examine the results. In both cases you needed a temporary buffer to hold the output/input. Often in Java a StringBuffer is used for this but that is (1) inefficient/slow, and (2) character oriented when you need bytes. The common practice of storing bytes in characters causes a lot of confusion and bugs. Programmers tend to want to use String for everything since they are the basic building block so of any programs, but constructing a long string programmatically, by building substring and putting them together in a recursive way is very inefficient, and the string is copied many times in the process.

To compose something from strings in memory, use this approach:

 MemFile mf = new MemFile();
 Writer w = mf.getWriter();
 w.write("This is the first line\n");
 w.write("This is the second line\n");
 w.write("This is the third line\n");
 // then use getReader() or getInputStream() to pass to a method that consumes
 // the file as if it was a stream.
 

Threading: MemFile should be used and accessed only from a single thread. Program should input everything to the mem file, and then read everything. Helper classes for InputStream and OutputStream will not necessarily return the right values if input is done at the same time as output.

Why not use a StringBuffer? Because a StringBuffer is optimized for fast conversion to a string, and to do this it keeps all the characters in a single contiguous array. While you are filling the buffer, if it runs out of room, it allocates a bigger contiguous array, and then copies the characters from the old buffer to the new buffer. This can happen multiple times. MemFile will never do this, because it does not require the bytes to be in a single contiguous array.

What if you need a String to pass to a method. You can construct a string in the normal way: Create a StringBuffer, create a StringBufferWriter, and ask the MemFile to write the entire contents to that. But once you start using streams correctly, the need to convert them to strings is almost elminated.

Author: Keith Swenson Copyright: Keith Swenson, all rights reserved License: This code is made available under the GNU Lesser GPL license.

Constructor Summary

Constructors

Constructor and Description
MemFile()

Method Summary

Modifier and Type	Method and Description
void	addPartial(byte[] buf, int pos, int len) copies the specified number of bytes from the byte array and adds it to the file.
void	adopt(byte[] buf) Takes the byte array and adds it to the file.
void	clear() Gets rid of all stored contents and clears out memory ready to receive new content.
void	fillWithFile(java.io.File file) Reads all bytes from File and stored the entire contents in memory.
void	fillWithInputStream(java.io.InputStream in) Reads all bytes from the passed in InputStream and stored the entire contents in memory.
void	fillWithReader(java.io.Reader in) Reads all character from the passed in Reader and stores the entire contents in memory.
java.io.InputStream	getInputStream() Returns an input stream which may be read from in order to read the contents of the memory file.
java.io.OutputStream	getOutputStream() Returns an output stream which may be written to in order to fill the memory file.
java.io.Reader	getReader() Returns a Reader which may be read from in order to read the contents of the memory file, assuming that the file is in UTF-8 encoding.
java.io.Writer	getWriter() Returns a Writer which may be written to in order to fill the memory file.
void	outToFile(java.io.File file) Writes the entire contents of the memory file to the file name passed in
void	outToOutputStream(java.io.OutputStream out) Writes the entire contents of the memory file to the OutputStream passed.
void	outToWriter(java.io.Writer w) Writes the entire contents of the memory file to the Writer that is passed.
java.lang.String	toString() Returns the entire contents of the MemFile as a single string.
int	totalBytes() Returns the number of bytes that the MemFile currently is holding.
int	totalChars() Returns the number of characters that the MemFile currently is holding.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MemFile

public MemFile()
        throws java.lang.Exception

Throws:

java.lang.Exception

Method Detail

clear

public void clear()

Gets rid of all stored contents and clears out memory ready to receive new content.

fillWithFile

public void fillWithFile(java.io.File file)
                  throws java.lang.Exception

Reads all bytes from File and stored the entire contents in memory. The file is expected to be UTF-8 encoded.

Throws:

java.lang.Exception

fillWithInputStream

public void fillWithInputStream(java.io.InputStream in)
                         throws java.lang.Exception

Reads all bytes from the passed in InputStream and stored the entire contents in memory. Closes the input stream.

Throws:

java.lang.Exception

fillWithReader

public void fillWithReader(java.io.Reader in)
                    throws java.lang.Exception

Reads all character from the passed in Reader and stores the entire contents in memory. Closes the reader.

Throws:

java.lang.Exception

outToOutputStream

public void outToOutputStream(java.io.OutputStream out)
                       throws java.lang.Exception

Writes the entire contents of the memory file to the OutputStream passed. Does not close the output stream.

Throws:

java.lang.Exception

outToWriter

public void outToWriter(java.io.Writer w)
                 throws java.lang.Exception

Writes the entire contents of the memory file to the Writer that is passed. Does not close the writer.

Throws:

java.lang.Exception

outToFile

public void outToFile(java.io.File file)
               throws java.lang.Exception

Writes the entire contents of the memory file to the file name passed in

Throws:

java.lang.Exception

getInputStream

public java.io.InputStream getInputStream()

Returns an input stream which may be read from in order to read the contents of the memory file.

getReader

public java.io.Reader getReader()
                         throws java.lang.Exception

Returns a Reader which may be read from in order to read the contents of the memory file, assuming that the file is in UTF-8 encoding.

Throws:

java.lang.Exception

getOutputStream

public java.io.OutputStream getOutputStream()

Returns an output stream which may be written to in order to fill the memory file. Adds to the end of whatever is currently in memory, so use "Clear" if you want to start with an empty memfile. Holds a buffer, so be sure to flush and/or close to get the complete value into the mem file.

getWriter

public java.io.Writer getWriter()
                         throws java.lang.Exception

Returns a Writer which may be written to in order to fill the memory file. Adds to the end of whatever is currently in memory, so use "Clear" if you want to start with an empty memfile. Only supports UTF-8. Holds a buffer, so be sure to flush and/or close to get the complete value into the mem file.

Throws:

java.lang.Exception

adopt

public void adopt(byte[] buf)

Takes the byte array and adds it to the file. NOTE: the actual object is retained, so if you modify the contents of this buffer you will modify the file. Do NOT reuse the buffer after passing it to this routine.

totalBytes

public int totalBytes()

Returns the number of bytes that the MemFile currently is holding.

totalChars

public int totalChars()

Returns the number of characters that the MemFile currently is holding. Caution, the result is accurate, but this requires scanning and deciding all the bytes to determine when multibyte sequences add up to only a single character.

addPartial

public void addPartial(byte[] buf,
                       int pos,
                       int len)

copies the specified number of bytes from the byte array and adds it to the file. It is OK to use the buffer for other purposes after this.

toString

public java.lang.String toString()

Returns the entire contents of the MemFile as a single string. It first figures out how many characters there are, and then allocates a single StringBuffer of the right size, puts the characters into it, and returns the string. It does this copying every time you call it, so be frugal.

Overrides:

toString in class java.lang.Object