compatibility with ruby-progressbar (and csv)

[dorianmariecom]

require 'parallel'; require 'ruby-progressbar'; progress_bar = ProgressBar.create; Parallel.each(1..100) { |i| sleep(rand(2)); progress_bar.increment }

https://github.com/user-attachments/assets/278ed85c-caf3-42c3-9535-b7dde454e383

I also posted on ruby-progressbar repository https://github.com/jfelchner/ruby-progressbar/issues/196

I have some issues with csv as well, so I was thinking maybe it's something from parallel

[dorianmariecom]

>> require 'csv'; require 'parallel'; CSV.generate { |csv| Parallel.each([1, 2, 3]) { |element| csv << [element] } }
=> ""

[dorianmariecom]

works with threads, not with processes

[grosser]

Can't possibly work with processes since they are not connected to each other, but forked. To make this work either:

• Parallel.map(1..50, progress: "Doing stuff") { sleep 1 }
• Parallel.map(1..100, finish: -> (item, i, result) { ... progress bar stuff ... }) { sleep 1 }

[dorianmariecom]

thanks @grosser, that works for the progress bar, but not for the csv :(

is there any way to run csv generation in parallel?

also didn't know parallel had a progress bar built-in, that's nice

[grosser]

You can generate CVS in each process, either return them or write to separate files on disc, and then combine them after everything is done.

On Sun, Sep 1, 2024, 1:08 PM Dorian Marié @.***> wrote:

thanks @grosser https://github.com/grosser, that works for the progress bar, but not for the csv :(

is there any way to run csv generation in parallel?

also didn't know parallel had a progress bar built-in, that's nice

— Reply to this email directly, view it on GitHub https://github.com/grosser/parallel/issues/352#issuecomment-2323481270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACYZYI6I7BACJS6FZD2RLZUNX23AVCNFSM6AAAAABNOYIXAKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGQ4DCMRXGA . You are receiving this because you were mentioned.Message ID: @.***>

[dorianmariecom]

thanks, really smart by the way

i did this:

        result = headers_of(content) ? headers_of(content).to_csv : ""

        result + map(content) do |element|
          CSV.generate(headers: headers_of(content)) do |csv|
            csv << wrap(element)
          end
        end.join