search

ACDguide / BigData

Working with big/challenging data collections
https://ACDguide.github.io/BigData
Other
6 stars 5 forks source link

Oct restructure #81

Closed paolap closed 1 year ago

paolap commented 1 year ago

I re-organised the book following the Miro board. There might be a couple of points where I misinterpreted some of the comments, in some cases I left comments in the relevant diagram boxes, but I think the overall structure should be what expected.

I've added a "homeless" chapter under data, as there were a couple of bits of text that I think weren't listed in the restructuring. I believe all the other bits and pieces are there, except the basically empty files to add more computations examples for dask, machine learning and to deal with time axis, which were also not included in a diagram. As these were just placeholders I didn't add them at all, but I left a note in the diagram about them.

I tidied up the files so they're all in subdirectories for the respective sections.

Notably, the accessing data part is really quite messy, it will need a re-organisation of its own, I preferred just put the content and push this quickly, rather than spend a lot of time in details.

I strongly suggest to clone this and try to build it on your own laptop if you can, trying to compare it with the current main will be quite challenging. If/when we're happy to merge, I cloned the current main branch already as main-backup-221027, to make it even easier to retrieve it/compare if we want to later on.

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

paigem commented 1 year ago

This looks great @paolap! What a huge effort to put this restructure together. I will look things over and add some comments, but I propose we go through this restructured PR at our meeting later this week. We can decide together how the new sections look and where the extra text should be housed.

paolap commented 1 year ago

sounds a good plan, it took less than you might think as some of the restructuring Dougie suggested coincided with what I've already done before in my separate branch :-)

dougiesquire commented 1 year ago

Thanks @paolap! Apologies, I haven't had a chance to look at this in any detail yet. I like @paigem's suggestion to go through this together in our meeting. I'll definitely be able to take a good look through before then

paolap commented 1 year ago

I've applied the changes we discussed at the meeting, so this is ready to merge. I've started adding some text to the data structure section, as well as using existing bits, let me know what you think about it. There will be still quite a lot to fix in the netcdf page, which is now so long and is a combination of bits and pieces that used to leave in different spaces.

hot007 commented 1 year ago

For what it's worth I think tidying the netCDF page can be relatively low priority, as use of headings means it has a ToC on the right hand side which assists navigation/overwhelm.