Closed dorianmariecom closed 1 month ago
>> require 'csv'; require 'parallel'; CSV.generate { |csv| Parallel.each([1, 2, 3]) { |element| csv << [element] } }
=> ""
works with threads, not with processes
Can't possibly work with processes since they are not connected to each other, but forked. To make this work either:
Parallel.map(1..50, progress: "Doing stuff") { sleep 1 }
Parallel.map(1..100, finish: -> (item, i, result) { ... progress bar stuff ... }) { sleep 1 }
thanks @grosser, that works for the progress bar, but not for the csv :(
is there any way to run csv generation in parallel?
also didn't know parallel had a progress bar built-in, that's nice
You can generate CVS in each process, either return them or write to separate files on disc, and then combine them after everything is done.
On Sun, Sep 1, 2024, 1:08 PM Dorian Marié @.***> wrote:
thanks @grosser https://github.com/grosser, that works for the progress bar, but not for the csv :(
is there any way to run csv generation in parallel?
also didn't know parallel had a progress bar built-in, that's nice
— Reply to this email directly, view it on GitHub https://github.com/grosser/parallel/issues/352#issuecomment-2323481270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACYZYI6I7BACJS6FZD2RLZUNX23AVCNFSM6AAAAABNOYIXAKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGQ4DCMRXGA . You are receiving this because you were mentioned.Message ID: @.***>
thanks, really smart by the way
i did this:
result = headers_of(content) ? headers_of(content).to_csv : ""
result + map(content) do |element|
CSV.generate(headers: headers_of(content)) do |csv|
csv << wrap(element)
end
end.join
https://github.com/user-attachments/assets/278ed85c-caf3-42c3-9535-b7dde454e383
I also posted on ruby-progressbar repository https://github.com/jfelchner/ruby-progressbar/issues/196
I have some issues with csv as well, so I was thinking maybe it's something from parallel