Planned Features

The following is a list of planned features, in no particulat order.

GUI

Checkpoint

This is in essence the ability to re-run a query on a datasource, and process only the new data. This would require the engine to save some state and then add the new data to that state when executed on the same dataset, with the same query again.

More efficient join implementation


The current join implementation is not very efficient O(n x m). Find a way to improve the efficiency, maybe by attempting to build some sort of an on the fly index or indices

Tail

The ability to use logQuery as a tail utility. This would require quite a significant re-design of the datasource implementation

Additional output formatters

Output formatters for Graph

DISTINCT

This is possibly the easiest to implement.

Easy way to specify and add a new function, datasource or output formatter

Ideally, we would want to be able to annotate the class, drop the jar in the lib folder and the objectis automatically available

Stored transforms

The ability to specify a transform, and save it is a file, then referencing it as a USING to transform the input source

clear