Upload Data (2.0 API)

Concepts

The 2.0 lowermost upload REST API involves the following steps:

  1. Generate a short-living JWT session token using your account credentials
  2. Optionally (recommended) upload your data as Files. The fastest, most scalable, and most reliable formats are ORC/Parquet/Arrow formats.
  3. Create a visualization Dataset with any optional configuration (JSON) describing how to turn your data into a graph. If you didn't upload data in the previous step (recommended), you can do it here.

Several convenience methods and language-specific client libraries make it even easier to use.

Notes:

  • Graphistry's built-in notebook server: If you are using the REST API through the provided Jupyter Notebook, you will likely want to use the Docker internal network base path of http://nginx.
  • Trailing slashes: Take note of the use of trailing slashes ("/").
For examples, the documentation includes example curl calls. For Python examples, see PyGraphistry's ArrowUploader class.

Create dataset

Logo style follows the CSS filter specification and CSS gradients. Blend mode values follow the CSS mix-blend-mode specification. For setting additional style properties, see the URL API and how to set encodings.

The node_files and edge_files refer to the string file_id returned by /file/ API calls and the index of the table(s) in them. You can use them in two ways:

  • First/only table in the file: edge_files: [ "file_id123" ]
  • Arbitrary table #22 in the file: edge_files: [ ["file_id123", 22] ]

Route Method Headers Parameters Return
api/v2/upload/datasets/ POST Content-Type: application/json
Authorization: Bearer YOUR_JWT_TOKEN
{
  "node_encodings": {
    "bindings": {
      ?"node": str,
      ?"node_color": str,
      ?"node_label": str,
      ?"node_opacity": str,
      ?"node_size": str,
      ?"node_title": str,
      ?"node_weight": str
    },
    ?"complex": [ see complex section ]
  },
  ?"node_files": [ str | [ str, int ] ],
  "edge_encodings": {
    "bindings": {
      "source": str, 
      "destination": str,
      ?"edge_color": str,
      ?"edge_label": str,
      ?"edge_opacity": str,
      ?"edge_size": str,
      ?"edge_title": str,
      ?"edge_weight": str
    },
    ?"complex": [ see complex section ]
  },
  ?"edge_hypergraph_transform": [{
    ?"entity_types": [ str ],
    ?"direct": bool,
    ?"drop_edge_attrs": bool,
    ?"opts": {
      ?'EVENTID': str,
      ?'CATEGORIES': {
        str: [ str ]
      },
      ?'DELIM': str,
      ?'SKIP': [ str ],
      ?'EDGES': {
        str: [ str ]
      }
    }
  }]
  ?"edge_files": [ str | [ str, int ] ],
  "metadata": { ?[ see metadata section ] }
  "name": str,
  ?"description": str
}
        
{
  "data": {"dataset_id": str},
  "message": str,
  "success": bool
}
        
Input:

curl -s -L -X POST \
  -H "Authorization: Bearer my_generated_token" \
  -H "Content-Type: application/json" \
  -d'{"node_encodings": {"bindings": {}}, \
      "edge_encodings": {"bindings": { \
          "source": "s", "destination": "d"}}, \
      "metadata": {}, "name": "mytestviz"}' \
  http://localhost/api/v2/upload/datasets/
Output:

{
  "data": {"dataset_id": "my_generated_dataset_id"},
  "message": "Dataset created",
  "success": true
}
Input:

curl -s -L -X POST \
  -H "Authorization: Bearer my_generated_token" \
  -H "Content-Type: application/json" \
  -d'{"node_encodings": {"bindings": {}}, \
      "node_files": [ "my_fileid123" ],
      "edge_files": [ ["my_fileid456", 1 ] ]
      "edge_encodings": {"bindings": { \
          "source": "s", "destination": "d"}}, \
      "metadata": {}, "name": "mytestviz"}' \
  http://localhost/api/v2/upload/datasets/
Output:

{
  "data": {"dataset_id": "my_generated_dataset_id"},
  "message": "Dataset created",
  "success": true
}

Delete dataset

Route Method Headers Parameters Return
api/v2/datasets/<dataset_id> DELETE Content-Type: application/json
Authorization: Bearer YOUR_JWT_TOKEN
HTTP code 204
Input:

curl -s -L -X DELETE \
  -H "Authorization: Bearer my_generated_token" \
  -H "Content-Type: application/json" \
  http://localhost/api/v2/datasets/1234
Output:

204

Hypergraphs

The hypergraph transform gets enabled when using the parameter edge_hypergraph_transform.

The hypergraph transforms creates a node for every unique value in the entity_types columns (default: all columns). If direct=False (default), every row is also turned into a node.

The transform will dictate the src/dst edge columns based on whether direct=True (src/dst vs EventID/attribID). By default, the node ID column should be nodeID, unless you provide your own node table.

Ex: Connect all columns through the row: row -> employee, boss, subsidary, year_hired, ...

{
  "node_encodings": {"bindings": {"node": "nodeID"}},
  "edge_encodings": {"bindings": {"source": "EventID", "destination": "attribID"}},
  "edge_files": ["FILE_123"],
  "metadata": {},
  "name": "hyper_123",
  "edge_hypergraph_transform": [{}]
}

Ex: Connect employee, boss, and subsidary together (no rows), merge employee/boss nodes that use the same name, control edges creation, and do not show nulls

{
  "node_encodings": {"bindings": {"node": "nodeID"}},
  "edge_encodings": {"bindings": {"source": "src", "destination": "dst"}},
  "edge_files": ["FILE_123"],
  "metadata": {},
  "name": "hyper_123",
  "edge_hypergraph_transform": [{
    "entity_types": [ "employee", "boss", "subsidary" ],
    "direct": true,
    "drop_na": true,
    "drop_edge_attrs": true,
    "opts": {
      "CATEGORIES": {
        "person": [ "employee", "boss" ]
      },
      "EDGES": {
        "person": [ "boss", "subsidary" ],
        "subsidary": [ "boss" ]
      }
    }
  }]
}