Thursday, 12 September 2013

Convert large CSV files to JSON

Convert large CSV files to JSON

I don't mind if this is done with a separate program, with Excel, in
NodeJS or in a web app.
It's exactly the same problem as described here:
Large CSV to JSON/Object in Node.js
It seems that the OP didn't get that answer to work (yet accepted it
anyway?). I've tried working with it but can't seem to get it to work
either.
In short: I'm working with a ~50,000 row CSV and I want to convert it to
JSON. I've tried just about every online "csv to json" webapp out there,
all crash with this large of a dataset.
I've tried many Node CSV to JSON modules but, again, they all crash. The
csvtojson module seemed promising, but I got this error: FATAL ERROR: JS
Allocation failed - process out of memory.
What on earth can I do to get this data in a useable format? As above, I
don't mind if it's an application, something that works within Excel, a
webapp or a Node module, so long as I either get a .JSON file or an object
that I can work with within Node.
Any ideas?

No comments:

Post a Comment