RVY200765: Direct writing of data to the tables for Data Hub.

Some RayVentory queries in Data Hub have several million records. RayVantory Data Hub Connector works in multiple stages, reading the data into memory, then writing to a CSV file, then transferring the complete data via HTTP into REST API.

If both databases are on one machine, it is much more efficient to write the data directly.

It is possible to change the stored procedures in the "ManageSoft" database so that the stored procedures do not return data, but write directly to the tables for the Data Hub.


There are three SP can be adjusted:

• csp_aspera_connector_software_arp
• csp_aspera_connector_software_file
• csp_aspera_connector_software_generic_othersw

One example:

1. We are dropping the physical table in Data Hub
2. We insert the results of select into physical table
3. We are returning count of the table as dummy value to avoid collision with agent.


Additionally, we renamed the target DATASET NAME in tasks setting of Data Hub by adding "_direct", so the dummy data will be written into tables with _direct on the end.

If this is the case for you, please contact our consulting team for more support.