Skip to main content

File

Read a CSV, JSON or Parquet file by URL. The relation is memoized: it is fetched once.

Syntax​

{ "id": "n1", "op": "file", "name": "file", "x": 0, "y": 0, "params": {} }

One out of it starts at "sourcePort": "out".

Inputs​

This operator has no input ports.

Parameters​

ParameterKindDefaultChoices
urlurl""
formatmenu"csv"csv, json, parquet
headertoggletrue
namesstring""
nullstrstring""

Outputs​

PortType
outtable

Remarks​

File reads one CSV, JSON or Parquet file into a table in DuckDB, in the engine worker. The table is read once and kept: a change downstream never reads the file again, and changing the URL, the format or a CSV option reads it anew.

A URL that starts with @/ names a file in the project's data directory, so @/data.csv is the data.csv beside noodles.json. Any other URL is fetched by the browser, so a server on another domain has to allow cross-origin requests (CORS).

DuckDB detects each column's type from the file. The Header row, Column names and Null marker options apply to CSV only: turn Header row off and list Column names for a file whose first row is data. A JSON file is an array of objects, one row each.

To compute a value per row, add an Attribute Create after it, or a DuckDB SQL node for anything a query does better.

Examples​

A CSV stored with the project, its positions made into a column for a layer:

{
"nodes": [
{ "id": "n1", "op": "file", "name": "stations", "x": 0, "y": 0, "params": { "url": "@/stations.csv" } },
{ "id": "n2", "op": "attribute", "name": "position", "x": 300, "y": 0, "params": { "name": "position", "expression": "[lng, lat]" } }
],
"edges": [{ "id": "e1", "source": "n1", "sourcePort": "out", "target": "n2", "targetPort": "in" }]
}

Details​

  • Category: Data
  • Operator: file

Older projects​

Projects saved before documents used operator classes. These convert to file when they open: FileOp.