Realtime Database
What is it?
In a kdb+ tick system, at some point we are going to want to retrieve and query the data we have received throughout the day. This is the purpose of the RDB, the Realtime Database - a database containing all of the realtime updates today. The updates are stored in memory until the end of the day when they are written down to disk.
Basics
The RDB performs the following functions in a (vanilla) tick system:
Subscribe to the tickerplant to receive today's updates for all tables and all syms
Note: in many systems there are multiple RDBs which handle different categories of data
Replay any missed data on startup
Provide an in-memory database for today's data
Save today's data down to its relevant location at end of day
Note: in many systems this is handled by a Writedown process
Starting the RDB
When starting the RDB, the syntax is as follows:
q tick/r.q [host1]:port1[:usr:pwd] [host2]:port2[:usr:pwd]
Where host1/port1 are the host and port for tickerplant, and host2/port2 are the host and port for the HDB.
If no values are given, 5010 and 5012 will be used for the port numbers.
You can find a line-by-line breakdown of the r.q script here.
If the RDB starts after the Tickerplant has already started to receive and publish data, it will need to catch up by replaying the tickerplant log file. Once this has been replayed, it can start receiving updates from the Tickerplant.
End of Day
At the end of day, the RDB will save down the day's data to today's date partition. The end of day is called by the Tickerplant, which runs .u.end on all subscribers once a new day has started.
Within .u.end, the RDB calls .Q.hdpf, which is a function that is pretty much designed for the RDB. It saves down all tables present in the RDB by calling .Q.dpft for each, it then wipes the content of those tables from memory, and sends a reload message to the HDB.