weird json format of pets data through FTP

 
1
0
-1

I'm working on an app, try to use my account to download those pets data files through FTP, then write a server to update json data daily into firebase's database, but I just realize the data ( pets_n.json ) format is not pure json.

I believe json only allows one root, it should be like {"a":{...},"b":{...}} or [{...},{...}]

but what I got in those .json file are {...}{...}{...}, which is not recognizable by nodejs or IDE.

I'm wondering is this on purpose? And how do people deal with this? Maybe I should write my own parser to re-write into json format？  

Thank you very much

    CommentAdd your comment...

    3 answers

    1.  
      2
      1
      0

      cl,

      I believe the intent is for you to process each line of the data file as a JSON object, and not attempt to handle the entire dataset as a whole.

      If we were to have just one JSON root, you'd have to concatenate 10 or more files into one >650 MB file before you could process the JSON.

      Hope that explains it,

      Richard

      1. cl

        yeah I found out those objects were split by '\n' so I used stream/buffer to read lines and write into database.

        I was confused yesterday because I'm used to frontend stuff and I saw they are all .json files but inside not real json, perhaps '.txt' would be cleaner?

        Thank you for the help

      CommentAdd your comment...
    2.  
      1
      0
      -1

      I am also having trouble with this issue, but do not understand how to fix the invalid JSON files. It's easier for me to just upload the JSON files as a whole, so I was hoping someone could tell me how to fix the invalid JSON files so that I can just upload the files from the API as a whole. Thanks!

        CommentAdd your comment...
      1.  
        1
        0
        -1

        I just got access to those files today and was equally confused. I am glad I was able to find this article.

          CommentAdd your comment...