🔎

    Changelog (2024)

    2024-12-26🔗

    • Feature: Chart / Plotter node can now integate with the Report Generator node to include charts in reports.
    • Feature: Improved Chart / Plotter node default axes configuration.

    2024-12-23🔗

    • Bug Fix: Fixed an issue primarily impacting the Plotter node (but could affect any node which takes data at run time for configuration) where the dropdowns for Reference data columns was failing input validation until the graph was saved and loaded at least once.
    • Bug Fix: Fixed pause node behavior in a couple nodes which were using a previous approach to pausing at edit/runtime.

    2024-12-20🔗

    • Feature: Improved the visual style of nodes in the Graph Applet.
    • Feature: Completely redesigned the Chart / Plotter Node, for future interoperability with other nodes (e.g. Report Generator) and future decoupling visual dependencies from the node itself. Currently has basic support for Bar, Line, Scatter, and Pie charts.
    • Feature: Node > Manage > Logs now supports rich objects in addition to string messages. Objects are formatted for readability.
    • API Enhancement: Some core data types have been collapsed into the serde_json::Value type for simplicity and consistency.
    • API Enhancement: manifest.json now has a schema available at https://datacogs.net/public/export/manifest_schema.json.

    2024-12-15🔗

    • Feature: Added a new Report Generator node to generate text (e.g. HTML) reports with data received from the input ports. The report can be customized with a template in Tera (Jinja2) format.

    2024-12-13🔗

    • Feature: Added Aggregate Data node to aggregate data based on a grouping column and aggregation functions.
    • Feature: Source nodes added to the graph will automatically be in a paused state. This allows for more control when adding nodes while the graph is running, ease configuration, and ease connecting nodes at runtime.
    • Bug Fix: Cleaned up some issues with the Graph Applet and Orchestrator to improve start / stop behavior.
    • Bug Fix: Nodes with required outputs will now pause when they have no downstream nodes to receive the output.
    • Bug Fix: Enhancements to Chart node / applet, in particular with default configuration.
    • API Enhancement: manifest.json files can now have comments and trailing commas.

    2024-12-11🔗

    • Major enhancements to the Graph (complete Orchestrator rewrite):
      • Feature: Live Graph - Rather than only running to the selected node, the entire graph now runs when possible based on "Source" nodes. This allows for more complex data processing pipelines and more comprehensive visualizations.
        • Nodes can now branch without having to terminate into a single node.
        • Source nodes bring data into the graph, and the graph runs to the extent possible based on active source nodes. Nodes not connected to an active source node won't run.
        • Nodes are now automatically started when they receieve input.
      • Feature: Nodes can now be paused/unpaused at runtime, and nodes can pause themselves to wait for user configuration. Nodes can asynchronously await during pause. Upstream nodes can asynchronously block when a downstream node is paused, so they can either continue work buffering data or pause themselves.
        • Paused nodes allow input, but hold input responses and hold output until unpaused. Otherwise they can interact with other APIs such as reading files and monitoring configuration changes. This allows for dynamic runtime configuration, such as generating dropdowns for selecting columns in tabular data.
      • Performance: Nodes and their asssociated Web Workers are now lazily loaded and automatically unload. This decreases memory usage and improves performance.
      • Performance: Rather than one web worker per node, nodes are automatically grouped and share a single web worker when they are the same module (executable code). This reduces the number of web workers and improves performance.
      • API Enhancement: Simplified and consoliated some Graph and Node API commands. Improved type safety. Renamed some types for clarity based on current architecture.
      • API Enhancement: Node lifecycle / run state is now more clearly defined, in particular with the UserActionRequired state.
      • Bug Fix: Graph Applet Canvas is now directly in control of the Orchestrator object, which fixes an issue where sometimes the Orchesrtator would crash due to improper handling of the mutex.
    • Minor Enhancements:
      • Hovering over tab title now shows full title in a tooltip.
      • Bug Fix: Editor Applet - Removed spurrious error messages.
      • Orchestrator now has a direct connection to the Storage / Shared Worker. This allows for more efficient data sharing between nodes and the Graph without having to route to the Graph Applet (iframe) first.
      • Slight optimizations to the graph file format to not save empty or null values.

    2024-11-24🔗

    • Feature: Added initial implementations for a Chart Node and Chart Applet for visualizing data in various chart types.
    • API Enhancement: Added a crate for reusable UI components (so far has Tabs, Accordion, Toggle Switch, Sidebar Layout) to be used in Applets.

    2024-11-21🔗

    • Bug Fix: Fixed an issue with the Service Worker that caused it to not update in some cases.
    • Bug Fix: Decreased the size of applets due to all wasm_bindgen snippets incorrectly being included in all applets.
    • Feature: Standard URL links in the Global Menu bar (top-right) are now standard anchor tags. Useful for opening Documentation in a new tab or getting the url directly.
    • API Enhancement: Nodes can now optionally specify a "View" when attempting to open a compatible Applet.
    • Feature: Added ability to filter nodes by inputs, outputs, and tags in the Node Library.
    • Bonus: Root page is now a little less boring.

    2024-11-20🔗

    • Feature: Added basic search/filter functionality to the Node Library.

    2024-11-19🔗

    • Bug Fix: Fixed a regression affecting the SharedWorker which caused it load several times, triggering refresh issues, breaking folder monitoring, and causing some nodes to fail to run unless called multiple times.
    • Bug Fix: Fixed some issues with the Read File node and Read CSV node. Read CSV now supports reading a stream of files on the input port.
    • Feature: Implemented a new Read Files from Folder node, which streams files to the output port.

    2024-11-18🔗

    • Feature: Applets can now set a status on their enclosing Panel to indicate Modified, Loading, or Saving states.
    • Bug Fix: Rune node modified to work with new node connection API. Does not yet have Arrow support.

    2024-11-14🔗

    • Improved Applet and other server-hosted artifact download latency by parallelizing cache.
    • Improved panel / iframe loading indicators.

    2024-11-13🔗

    • Began work on documentation, hosted at https://docs.datacogs.net. Also integrated with the Datacogs platform for easy access to documentation while using the platform, available from the "Documentation" button in the top-right corner.

    2024-11-10🔗

    • Feature: Basic Model Performance Metrics node for building confusion matrices to evaluate model performance.
    • Many bug fixes and major performance enhancements in Graph applet and all graph nodes.

    2024-10-23🔗

    • Feature: Added several new nodes for working with training and predicting with models using Linear Regression, Logistic Regression, and Decision Trees.
    • Feature: Added a new node with example data sets for testing and training machine learning models.
    • Feature: Added several data processing pipeline nodes such as Feature Scaling, Normalization, and Encoding.
    • Feature: Various improvements to APIs, such as enhanced handling and control over file saving behavior, and improved data type management for node connections.
    • Bug Fix: Major overhaul of the BufferInfo mechanism (used to share data between nodes). Previously occasionally there was a race condition where some nodes would fail to read data before the producing node would overwrite it.

    2024-09-05🔗

    • Rune Support: New Rune Code Node for running custom Rune code directly within a node
    • Rune Extensions: Basic limited support for running extending Rune with custom modules that can be written in other languages, such as Rust, and compiled to WebAssembly. This provides more complex capabilities to all Rune scripts. Currently support custom Structs, Functions, and Enums including variant-based enums which work with Rune's pattern matching.
    • Improved code editing within nodes, and option to split into a separate editor panel.

    2024-08-27🔗

    • Can now open CSV and Arrow files (already uploaded/saved in Datacogs) directly to view them. Double-click a CSV in the File Browser to view it.
    • Switched to the Apache Arrow format for data storage and processing
    • Nodes can now have a configurable number of inputs and outputs
    • New Python Code Node for running custom Python code directly within a node
    • New Summary Statistics Node calculates various statistics about an input data stream
    • New Query Node enables interacting with data streams as if they were SQL databases using SQL-like queries

    2024-08-13🔗

    • Improved Graph Node Python API to support communication with other nodes
    • Reworked Graph user interface with improved settings integration
    • Basic keyboard navigation support in the File Browser applet

    2024-08-09🔗

    • Text Editor can now open and save files. Currently to open a file, drag and drop it into the Text Editor applet.
    • Graph nodes can now render custom HTML or load an applet directly within the node component on the graph
    • Partial Python support. Nodes can be written in Python, but can't communicate with other nodes yet.