@yanick - I am trying to create a shared LWP::UserAgent, which has the needed SessionID (in a cookie file) created from the login, to generate and download many (over 100K+) worth of data records (yes each it own distinct URL in the api). I was looking to have this LWP::UserAgent (same instance) referenced in each fork but getting some odd results trying to do this.
I can successfully login, but when I start the forking after it has retrieved it list of URL's from the API end point, it gets back 500 errors, which when I turn off forking (comment out the forking code) all seems to works as expected.
Not sure if you had an experience with that, and my suspicion is that forking the a single reference of LWP::UserAgent could be problematic.
@yanick - I am trying to create a shared LWP::UserAgent, which has the needed SessionID (in a cookie file) created from the login, to generate and download many (over 100K+) worth of data records (yes each it own distinct URL in the api). I was looking to have this LWP::UserAgent (same instance) referenced in each fork but getting some odd results trying to do this.
I can successfully login, but when I start the forking after it has retrieved it list of URL's from the API end point, it gets back 500 errors, which when I turn off forking (comment out the forking code) all seems to works as expected.
Not sure if you had an experience with that, and my suspicion is that forking the a single reference of LWP::UserAgent could be problematic.
Thoughts?