Definition of point file

[joelfiddes]

I think need to have a tighter definition of points file eg have discovered:

1. x,y can not be columns 1 and 2 (indexing starting at 1!) or else x is dropped as an index (kwargs arg to open csv)
2. index column in position 1 must be python indexed starting at 0,1,2,3 etc
3. first column cannot be a string (as in Finse example? This confuses me how it works)

[ArcticSnow]

By default pandas infers the index column is not present in the csv file:

In [2]: pd.read_csv('inputs/dem/station_list.csv')
Out[2]: 
               Name stn_number  latitude  longitude              x             y
0        Finsevatne    SN25830   60.5938     7.5270  419320.867306  6.718448e+06
1    Fet-I-Eidfjord    SN49800   60.4085     7.2798  405243.856318  6.698143e+06
2       Skurdevikåi    SN29900   60.3778     7.5693  421114.679132  6.694343e+06
3         Midtstova    SN53530   60.6563     7.2755  405730.301715  6.725742e+06
4  FV50-Vestredalen    SN53990   60.7418     7.5748  422296.164019  6.734872e+06
5       Klevavatnet    SN53480   60.7192     7.2085  402259.379227  6.732844e+06

So I see no problem.

[ArcticSnow]

and another quick test:

In [3]: pd.read_csv('inputs/dem/test.csv')
Out[3]: 
               x             y
0  419320.867306  6.718448e+06
1  405243.856318  6.698143e+06
2  421114.679132  6.694343e+06
3  405730.301715  6.725742e+06
4  422296.164019  6.734872e+06
5  402259.379227  6.732844e+06

In [4]: cat inputs/dem/test.csv
x,y
419320.867306002,6718447.86246835
405243.856317655,6698143.36494597
421114.679132306,6694343.36865902
405730.30171528,6725742.26010349
422296.164018722,6734871.61164008
402259.379226592,6732844.21093029

[joelfiddes]

thanks for the tests - I think what Im seeing maybe somehow ide (pycharm) specific:

Here is a a test.csv:

x,y,field_1,lat,lon,field_4,field_5 546622.379993986,3908941.01586193,1,35.322611,75.51294,,chunda 557622.216698521,3897911.54259219,2,35.22259,75.63318,8770,insitu 557939.463619031,3899389.74605043,3,35.2359,75.63677,8784,insitu 545396.542472077,3909413.75865801,4,35.32693,75.49948,9047,insitu 545331.174306277,3909579.78901023,5,35.32843,75.49877,9073,insitu 545895.939996163,3908977.1003069,6,35.32297,75.50495,8946,insitu 545783.497343643,3914639.43398378,7,35.37403,75.50403,7200,insitu 555367.284102864,3892742.65752312,8,35.17611111,75.60805556,13612.2,gmrc

if i do outside of ide (ipython):

In [8]: pd.read_csv("test.csv") Out[8]: x y field_1 lat lon field_4 field_5 0 546622.379994 3.908941e+06 1 35.322611 75.512940 NaN chunda 1 557622.216699 3.897912e+06 2 35.222590 75.633180 8770.0 insitu 2 557939.463619 3.899390e+06 3 35.235900 75.636770 8784.0 insitu 3 545396.542472 3.909414e+06 4 35.326930 75.499480 9047.0 insitu 4 545331.174306 3.909580e+06 5 35.328430 75.498770 9073.0 insitu 5 545895.939996 3.908977e+06 6 35.322970 75.504950 8946.0 insitu 6 545783.497344 3.914639e+06 7 35.374030 75.504030 7200.0 insitu 7 555367.284103 3.892743e+06 8 35.176111 75.608056 13612.2 gmrc

no problem.

BUT, if I do in PyCHARM i get the equiv of this:

In [7]: pd.read_csv("test.csv", index_col = 0) Out[7]: y field_1 lat lon field_4 field_5 x
546622.379994 3.908941e+06 1 35.322611 75.512940 NaN chunda 557622.216699 3.897912e+06 2 35.222590 75.633180 8770.0 insitu 557939.463619 3.899390e+06 3 35.235900 75.636770 8784.0 insitu 545396.542472 3.909414e+06 4 35.326930 75.499480 9047.0 insitu 545331.174306 3.909580e+06 5 35.328430 75.498770 9073.0 insitu 545895.939996 3.908977e+06 6 35.322970 75.504950 8946.0 insitu 545783.497344 3.914639e+06 7 35.374030 75.504030 7200.0 insitu 555367.284103 3.892743e+06 8 35.176111 75.608056 13612.2 gmrc

where index is set to x if there is no index column - somehow the ide has a default **kwargs set. If i hover the mouse you see this:

I need to continue to investigate......

[ArcticSnow]

Yes this seems to be related to PyCharm. Or do you use the same VE in both cases? Because there is not default value for kwargs and should be None as default. Weird stuff...