r.q

Note: This page goes through r.q line-by-line. For a higher level view of the logic involved in the RDB - see RDB.

What is it?

r.q is the main script used by the RDB process and handles some key functionality such as replaying the logfile. Let's break it down:

Changelog and initialisation

The first two lines are comments. The first line provides a sample startup syntax for an RDB. To start up an RDB the r.q script must of course be used, and the connection details for two processes - the tickerplant and the HDB. The second line provides the changelog (one change since 2008!)

Action 1 - sleep 

if[not "w"=first string .z.o;

  system "sleep 1"

];

The first action is to sleep for one second if the current operating system is not Windows. The one second sleep is to allow ample time for the tickerplant to start, as an error would be encountered if trying to connect to the TP before it had fully started. This does not seem to be an issue on Windows.

Action 2 - define 'upd' function

upd:insert;

Remember that the 'upd' function is the standard function called on any q process when it is receiving data. The standard inputs are table name and table data. For the vanilla RDB, we merely want to insert the data into the table in memory, e.g. `trade insert (data).

Action 3 - define .u.x

.u.x:.z.x,(count .z.x)_(":5010";":5012");

In the RDB, .u.x is a variable in the .u namespace containing the port details for the tickerplant and HDB. Due to the nicely written code here, if one or neither of these are assigned on startup, they will be given default values of 5010 and 5012 respectively. If more than two ports are referenced on startup, they will also be added to .u.x.

Define Functions

The rest of the script contains a number of function definitions. Note that each function is contained on one line. This can be difficult to read - let's format the code for easier readability as we go. 

Action 4 - define .u.end

.u.end is the end of day function called on the RDB by the TP (see u.q). It has one input of the date that has just ended.

  t:tables`.;

Assign a list of all tables in the global namespace to variable t.

  t@:where `g=attr each t@\:`sym;

Filter the list of tables to only include those which have the grouped attribute on the sym column. This might look a bit daunting but it's quite simple:

@\:`sym

Get the sym column for each table in t.

where `g=attr each t@\:`sym

Find which of those columns have the grouped attribute applied.

t@:where `g=attr each t@\:`sym;

Re-assign t to only contain those tables (this is the same as 't:t where ..').

Moving on:

  .Q.hdpf[`$":",.u.x 1;`:.;x;`sym];@[;`sym;`g#] each t;

Run .Q.hdpf which takes the following inputs: .Q.hdpf[historicalPort; directory; partition; `p#field].

• historicalPort

  • The second item in .u.x i.e. HDB port

• directory

  • Current directory (changed in .u.rep)

• partition

  • Input x i.e. the date that has just ended

• `p#field

  • `sym as the column on which to apply the parted attribute.

As clearing out the table causes the grouped attribute to be removed - re-apply it to the sym column for all tables.

Action 5 - define .u.rep

.u.rep is the function used to initialise schemas and replay data from the tickerplant. When examining .u.rep, it is important to know what the two inputs will be. As you can see below in Action 6, the first input to .u.rep is the output from running .u.sub on the tickerplant. That is broken down below, but suffice to say, input x is a list of (tableName; emptySchema) pairs. The second input then is a two item list containing the values of .u.i and .u.L in the tickerplant - i.e. number of log messages in the log file and the log file location.

  (.[;();:;].)each x;

This strange looking line of code is simply going through each item in list x and setting the empty schema to the table name. 

  if[null first y;

    :()

  ];

.u.rep is the replay function. If the tickerplant has returned a null value for .u.i it implies there is some kind of problem and a replay will not  be possible. In this case, return early.

  -11!y;

Replay the tickerplant logfile. Input y at this stage contains (.u.i; .u.L) so when we use -11! with these inputs it will replay the logfile up to that point. 

  system "cd ",1_-10_string first reverse y

Change directory to the HDB directory. By default this is taken from the tickerplant logfile location, however the below line of code tells us that if our HDB dir is anywhere else, we should hardcode it in this script:

/ HARDCODE \cd if other than logdir/db

Action 6 - connect and subscribe to tickerplant

.u.rep .(hopen `$":",.u.x 0)"(.u.sub[`;`];`.u `i`L)";

This is the line of code that kicks the whole thing off and invokes all of the functions defined in the script. There's quite a lot going on here so let's break it down.

.u.rep . (...)

The last action that will be performed in this line of code is to run .u.rep with two inputs. This was explained above in Action 5. Now let's see how we get those inputs...

(hopen `$":",.u.x 0)"..."

The first item in .u.x contains the handle to the tickerplant. Open a handle to the tickerplant and run the following:

(.u.sub[`;`];`.u `i`L)

There are two queries being sent here. The result of the first query becomes the first input to .u.rep and the result of the second query becomes the second input to .u.rep.

1. Run .u.sub with backtick as both inputs i.e. subscribe to all syms across all tables. The final output of .u.sub (see u.q) is a list of (tableName; emptySchema) pairs. This then becomes the first input to .u.rep.

2. Get the values in .u.i and .u.L. These two items become the second input to .u.rep.

Comments