About: Method Compatibility

Communication within a process based on Semys works with the help of the messaging system for both client-server relations within the process and between the processes.

Modules, based on Semys Arch, define their public interface with the help of messages. They somewhat resemble a simple C header file, specifying public function signatures.

If a certain function extends its possibilities, typically extra parameters are defined, but the other (older) modules that use the interface still should be able to use the 'original' variant of a function.
It is not always possible to update all (relevant) modules, because then the system probably has to be shut down totally (a scenario sometimes not allowed) or the dependancies are simply not known because the other module was written by someone else.

Using the analogy of the C header, a second function with the new parameters is defined. The first function variant is implemented simply calling the new variant with some default values for the new parameters.
For a message object, making a new variant available is done by simply adding the new 'parameters' as extra members, with their respective default values initialized in its constructor.

As an example, we define a function that compresses a file asynchronously which becomes extended to allow the user to specify a compression algorithm (which previously was always the "simple" algorithm):

void StartCompression(char const *fileName)
{
  StartCompressionVer2(fileName, "simple");
}

void StartCompressionVer2(char const *fileName, char const *algorithm)
{
  /* notify compression thread and return */
}

Accordingly, the (simplified) message equivalent:

class MsgStartCompression: public Message
{
  MsgStartCompression() { mAlgorithm = "simple"; }

  string mFileName;
  string mAlgorithm;
}

The message contains both variants: The new implementation uses the algorithm member, an old one simply ignores it and still turns out right.

This 'Method Version Compatibility' has to be mirrored when messages have to become serialized to be transported to another process:

The encoder writes a version number into the stream before writing the members sequentially. The first variant writes '1', the second '2' and so on.
The decoder reads the version number and then the members that were known to this version.

Note that the version number is only used during serialization. The user of a message does not need it. Compare to the example with functions above: The new variant doesn't know what called it.

This way the new implementation is compatible with both new and old clients using the same message.

Note that typically parameters (members) are only extended and rarely removed. If still a member becomes obsolete, it (or at least a similar dummy) must stay in the list of serialized fields.

This form of compatibility also helps in the backward signalization. Consider the example compression function notifies the original caller about completion.
First, it only notified the compressed size, but later implementations also notify the CRC value that was generated.

typedef void (*OnCompressionCompleted)(uint32 size);
typedef void (*OnCompressionCompletedVer2)(uint32 size, uint32 crc);

Using traditional function pointers, things will become complicated and need extra management overhead so that the compressor knows which version to call at the requester.
Using messaging, it stays simple:

class MsgCompressionCompleted: public Message
{
  MsgCompressionCompleted() { mSize = 0; mCRC = 0; }

  uint32 mSize;
  uint32 mCRC;
}

The new compressor doesn't have to care about which members the original requester knows or doesn't know, it simply fills them all. An old client on the other hand doesn't need to bother about new members - it didn't need them previously and doesn't know what to do with them anyway.

As a conclusion, versioning the serialized bitsream enables clients and servers both old and new to be connected in any way. (Note that even the scenario of a new client with an old server does happen - How, is an exercise for the reader.)

Note that this chapter describes automatic compatibility between different methods of the same function - A scenario with a requester that wants to start a function that simply isn't implemented at all must be handled in a different way.

Sometimes it is a thin line between another method of the same function or a new function at all. The new version has to be chosen carefully not to make its use a new function.

Experience showed that extensions are typically more detailed methods of the same function. None the less, this has to be considered as well.

The serialization process has to be slightly extended to consider all possibilities of a message exchange:

After writing the version number, the encoder first reserves an integer in the bitstream which is then filled with the actual size of the encoded block.
The decoder then reads the members as best as it knows and skips any unknown fields according to the length information.

Skipping the unknown members makes no sense at first, but does so when considering the full implementation:

The idea about the messages is that they can be extended by inheritance (they already need to be for the first specialization). So, while two layers are always present (one inheritance), three or even four are also possible.

Since every layer within a message typically serializes members, they also might be extended at one time. If, for example, the basic message class would add a new member, the old inheritance implementation would start to decode its first member at the position of the new basic one if the base didn't properly align the offset in the bitstream.

As an example, consider a module that implements file operations. The common thing about them is that every function refers to a file. This file name is the member of an 'abstract' message and all commands inherit from this one.
A later extension to these operations is that they all require a user reference for security checks.

Multiple message layers are not common, but this example is apropriate enough.

What follows are two examples from a new client that requests a 'Rename' operation at an old server. The first example is wrong as it does not respect skipping the unknown members, whereas the second does and is right.

Wrong; the decoded mFileNameNew is misaligned and will end up containing a user name:

New Encoder: string mFileName; string mUserName; string mFileNameNew;	Old Decoder: string mFileName; string mFileNameNew;

Right; the unknown members are skipped and all known ones are set properly

New Encoder: string mFileName; string mUserName; string mFileNameNew;	Old Decoder: string mFileName; (skipped) string mFileNameNew;

For reference, the described classes (simplified):

class MsgFileCommand: public Message
{
  MsgFileCommand() { }

  string mFileName;
  string mUserName; // Uses default permissions if empty
}

class MsgFileCmdRename: public MsgFileCommand
{
  MsgFileCmdRename() { }

  string mFileNameNew;
}

Generally described: Everything that follows a certain block of information must still be able to decode itself, even if the block became extended (changed).
This does not only apply to messages, but also to encoded state data.

Note that this abstract description also includes paragraph 09 from above.

Goto: Main Page; This page is part of the Semys software documentation. See About: Documentation for details.