rocketlaunchr / dataframe-go

DataFrames for Go: For statistics, machine-learning, and data manipulation/exploration
Other
1.19k stars 95 forks source link

add Input/Output to csv and json #5

Closed vlamai closed 5 years ago

vlamai commented 6 years ago

function works. But tests fails, i don't know why

        f,_ := os.Create("test.csv")
    defer f.Close()
    s1 := dataframe.NewSeriesInt64("day", nil, 1,2,3)
    s2 := dataframe.NewSeriesString("sales", nil, "a","b","q,w")
    df := dataframe.NewDataFrame(s1, s2)
    df.ToCSV(f,dataframe.CSVOptions{
        Comma:            ',',
        Comment:          0,
        TrimLeadingSpace: false,
        LazyQuotes:       false,
    })

        f,_ = os.Open("test.csv")
        defer f.Close()
    df1,_ := dataframe.FromCSV(f)
    fmt.Println(df1)
        f,_ := os.Create("test.json")
        defer f.Close()
    s1 := dataframe.NewSeriesInt64("day", nil, 1,2,3)
    s2 := dataframe.NewSeriesString("sales", nil, "a","b","q,w")
    df := dataframe.NewDataFrame(s1, s2)
    df.ToJSON(f)
    f.Close()

    f, _ = os.Open("test.json")
    defer f.Close()
    df1, _ := dataframe.FromJSON(f)
    fmt.Println(df1)

JSON import works only with [][]string file structure

rocketlaunchr-cto commented 6 years ago

Great work @kzoper I will look at this next weekend. However, my current thinking is changing the import package into io and having your functions inside that package rather than trying to replicate the pandas library function for function.

The reason why I am leaning that way is that the core functionality should be kept separate from enhancement functionality. Some users won't be using these import/export features so by including them in the main package will just increase the binary. Go files are already quite large.

rocketlaunchr-cto commented 6 years ago

What's the exact error you get when tests fail?

tappoz commented 5 years ago

Is there any news on this CSV/JSON in/out? I just found out the unexpected panic on the empty function body here: https://github.com/rocketlaunchr/dataframe-go/blob/master/imports/imports.go#L76 That's useless so I am wondering if there is any plan to merge/harmonize the code in here? Thanks

pjebs commented 5 years ago

I guess it card be too hard to implement a json import. I'll give it a try within a month.

It will have to use the json Decoder instead of unmarshaler however.

rocketlaunchr-cto commented 5 years ago

@tappoz json importing has been implemented. Check the json-imports branch to give me feedback before I merge it to master.

rocketlaunchr-cto commented 5 years ago

importing and exporting functionality has now been added.