gnudatalanguage / gdl

GDL - GNU Data Language
GNU General Public License v2.0
274 stars 61 forks source link

there is no IDL_IDLBridge equivalent #1857

Open ChunkyPanda03 opened 2 months ago

ChunkyPanda03 commented 2 months ago

I would like this feature however I don't think I could implement this

ChunkyPanda03 commented 2 months ago

we are destined to only use one thread for all of eternity :cry:

ChunkyPanda03 commented 2 months ago

@GillesDuvert was doing some work on this in purely idl/gdl I keep trying to use spawn but I just can't figure out a way to detach it from the main thread. I just don't think it is possible without writing some intermediary in C/C++ that handles the GDL instance what is your opinion on this?

GillesDuvert commented 2 months ago

@ChunkyPanda03 what IDL bridge?... As GDL is opensource, one does not necessary need one. Concerning SPAWN, I'm sure that /NOWAIT could be supported by GDL on all platforms with some work. People seem happy running many GDLs in parallel, eventually sharing common memory areas (although mmap is very recent and untested) . I understand that huge time-consuming data pipelines (sometimes unofficially :smile: ) use GDL in such manner.

ChunkyPanda03 commented 2 months ago

@GillesDuvert apologies I did not clarify IDL had a feature where you can automatically open another IDL interpreter in parallel this is known as IDL_IDLBridge, GDL lacks an equivalent feature (GDL_GDLBridge ?). This feature is great for obtaining multithreaded behavior in IDL. In my case I often need to do large analysis procedures which includes rendering png files which need to be off the main thread or the whole thing could take upwards of a couple days otherwise. This is also necessary in idl for window applications because the window will lock up if anything in the main thread is running.

ChunkyPanda03 commented 2 months ago

here is a link referencing the idl feature https://www.nv5geospatialsoftware.com/docs/idl_idlbridge.html

brandy125 commented 2 months ago

Yessss, this would be nice to have!

On 15. Jul 2024, at 16:26, ChunkyPanda03 @.***> wrote:

here is a link referencing the idl feature https://www.nv5geospatialsoftware.com/docs/idl_idlbridge.html

— Reply to this email directly, view it on GitHub https://github.com/gnudatalanguage/gdl/issues/1857#issuecomment-2229226250, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOC5K6CB4SCO4HTGMBST4JLZMQO6FAVCNFSM6AAAAABKZ2U4CWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRZGIZDMMRVGA. You are receiving this because you are subscribed to this thread.

GillesDuvert commented 2 months ago

interesting, and indeed useful --- probably just having GDL listen to external commands in its command loop , and adding a subprocess creation by fork() , plus some script for managing the IDL_IDLBridge object. Anybody interested? :smile:

ChunkyPanda03 commented 2 months ago

I sure am

GillesDuvert commented 1 month ago

nearly there... :smile:

GillesDuvert commented 1 month ago

@ChunkyPanda03 @brandy125 please test #1870 linux only --- works4me in the simple case I have tested. BUT OK in DEBUG mode only -- optimization make interprocess hang IPC needs to be more robust ...

GillesDuvert commented 1 month ago

seems ok now.

ChunkyPanda03 commented 1 week ago

Sorry got busy and missed this will have to give this a try

GillesDuvert commented 1 week ago

The current version MAY work. A better one using only POSIX messages is complete, needs to be made as a PR.