Open AlexZhouAA opened 3 years ago
Same issue
Same issue, occurred oct 6th
Since sessions seem to stay alive forever, maybe we can take a valid logged in session, steal the cookies and dump them into Kijiji-Repost-Headless?
Same issue
@encoad I like your idea. @ArthurG just point us in the right direction. I would be happy to fix this and create a PR
Same here. This tool has been incredibly unreliable since summer. Even when it did work, it would be very hit or miss. Doesn't help that sometimes kijiji itself takes a couple days to actually purge a deleted ad from the system before you're allowed to post it again.. Really hoping this can be fixed, used to work great last year!
Any hope of getting this going again?
I think the fundamental issue here is the way the login is done - it is trying to emulate a user on a web browser but once captcha is involved it makes this method much more difficult. Using something like the backend Kijiji API for login would be more robust and a less fail-prone method.
Since sessions seem to stay alive forever, maybe we can take a valid logged in session, steal the cookies and dump them into Kijiji-Repost-Headless?
That is a good idea. I just hardcoded session id cookie into the code and it's posting ads with no issue. If anyone wants to do the same thing, here is what I did.
kijiji_api.py
cookie_dict = {'ssid': 'REPLACE_WITH_YOUR_SESSION_ID'}
requests.utils.add_dict_to_cookiejar(self.session.cookies,cookie_dict)
To get your session id.
Save kijiji_api.py
, run it.
@JohnBakery, thank you for your elaborated procedures first. I followed your steps and got an error:
NameError: name 'resp' is not defined
Then I realized there're 8 resp = ...
statements within the file kijiji_api.py
.
Sorry, I'm not a python developer, do I need do the same to the rest?
Since sessions seem to stay alive forever, maybe we can take a valid logged in session, steal the cookies and dump them into Kijiji-Repost-Headless?
That is a good idea. I just hardcoded session id cookie into the code and it's posting ads with no issue. If anyone wants to do the same thing, here is what I did.
kijiji_api.py
- Comment out line #100 and #117. (Add # in front of those line)
- After line #117 add the following code
cookie_dict = {'ssid': 'REPLACE_WITH_YOUR_SESSION_ID'}
requests.utils.add_dict_to_cookiejar(self.session.cookies,cookie_dict)
To get your session id.
- Go to kijiji.ca
- Login into your account if you are not logged in.
- Press F12
- In FireFox: Go to Storage. In Chrome: Go to Application.
- Open up Cookies, select https://www.kijiji.ca, look for a cookie "ssid", double click on the value field and copy paste it into the code where it says REPLACE_WITH_YOUR_SESSION_ID.
Save
kijiji_api.py
, run it.
If we just do these things via selenium, then shut it down that should work fine. And since the session stays alive forever its just a one-time thing, so there could be a check for existance before having to trigger selenium stuff.
Either that or we add @JohnBakery's instructions to the readme which also isnt that bad tbh
@AlexZhouAA , my bad. In my code I actually commented out the whole block of code, but for the reply I only mentioned two lines to make the instructions easier, completely missing that resp
is used on another line in the login
function.
I edited my original post with the correct instructions. Do not comment out resp
anywhere else.
@JohnBakery, after I commented out the whole block as you instructed, I still got the error:
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\kijiji_api.py", line 124, in login raise KijijiApiException("Could not log in.", resp.text) NameError: name 'resp' is not defined
And then I commented out the following
if not self.is_logged_in()
raise KijijiApiException("Could not log in.", resp.text)
and try gain, I got error as
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\kijiji_api.py", line 251, in get_all_ads ads_json = json.loads(resp.text)
Please help & thanks for your time
@AlexZhouAA this means your session id value is incorrect. Either you didn't copy it completely (mine is over 300 characters long) or it expired. Make sure you when you login, select "Remember Me" option
@JohnBakery, you are absolutely right. I did copy the wrong ssid. Thank you so much. This time I got all text part posted successfully, but all images failed to be uploaded
Image #1 upload failed on try #1
Image #1 upload failed on try #2
Image #1 upload failed on try #3
Image #2 upload failed on try #1
Image #2 upload failed on try #2
Image #2 upload failed on try #3
Image #3 upload failed on try #1
Image #3 upload failed on try #2
Image #3 upload failed on try #3
Image #4 upload failed on try #1
Image #4 upload failed on try #2
Image #4 upload failed on try #3
Traceback (most recent call last):
File "~\Python36\lib\runpy.py", line 193,
in _run_module_as_main "__main__", mod_spec)
File "~\Python36\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 209,
in <module> main()
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 51, in main
args.function(args)
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 98, in post_ad
api.post_ad_using_data(data, image_files)
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\kijiji_api.py", line 238, in post_ad_using_data
raise KijijiApiException("Could not post ad.", resp.text)
kijiji_api.KijijiApiException: Could not post ad.
@AlexZhouAA Not sure why your images are failing. Maybe they are too big in size?
@JohnBakery, I have used the same scripts with the same images to post ads on kijiji.ca for a half year. It's not because of the image sizes for sure. Thank you so much for the helps
After reading @JohnBakery 's solution, I tried it and I managed to post an advertisement with pictures! It works!
Oddly enough when I used this app before every image would be uploaded on its first try but now it takes usually 2 tries for it to be uploaded. Addtionally, even when the ad has been uploaded successfully I still get this error::
Traceback (most recent call last):
File "~\Python36\lib\runpy.py", line 193,
in _run_module_as_main "__main__", mod_spec)
File "~\Python36\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 209,
in <module> main()
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 51, in main
args.function(args)
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 98, in post_ad
api.post_ad_using_data(data, image_files)
File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\kijiji_api.py", line 238, in post_ad_using_data
raise KijijiApiException("Could not post ad.", resp.text)
kijiji_api.KijijiApiException: Could not post ad.
It could be a false error where it raises an Exception when it should not. But, I can confirm that @JohnBakery 's solution is working and we could build upon his or her solution. So we are getting somewhere! ^_^
Since sessions seem to stay alive forever, maybe we can take a valid logged in session, steal the cookies and dump them into Kijiji-Repost-Headless?
That is a good idea. I just hardcoded session id cookie into the code and it's posting ads with no issue. If anyone wants to do the same thing, here is what I did.
kijiji_api.py
1. Add a new line after line #98 and type """ 2. Add a new line after line #118 and type """ 3. After line #119 add the following code
cookie_dict = {'ssid': 'REPLACE_WITH_YOUR_SESSION_ID'}
requests.utils.add_dict_to_cookiejar(self.session.cookies,cookie_dict)
To get your session id.
1. Go to kijiji.ca 2. Login into your account if you are not logged in. 3. Press F12 4. In FireFox: Go to Storage. In Chrome: Go to Application. 5. Open up Cookies, select https://www.kijiji.ca, look for a cookie "ssid", double click on the value field and copy paste it into the code where it says REPLACE_WITH_YOUR_SESSION_ID.
Save
kijiji_api.py
, run it.
To be clear, do I comment out the lines starting with: login_url ... and ending with: resp = ...
I did the above but some ads post properly and some have failures with image uploads. Thanks for the help.
After reading @JohnBakery 's solution, I tried it and I managed to post an advertisement with pictures! It works!
Oddly enough when I used this app before every image would be uploaded on its first try but now it takes usually 2 tries for it to be uploaded. Addtionally, even when the ad has been uploaded successfully I still get this error::
Traceback (most recent call last): File "~\Python36\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "~\Python36\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 209, in <module> main() File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 51, in main args.function(args) File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\__main__.py", line 98, in post_ad api.post_ad_using_data(data, image_files) File "~\Kijiji-Repost-Headless-master\kijiji_repost_headless\kijiji_api.py", line 238, in post_ad_using_data raise KijijiApiException("Could not post ad.", resp.text) kijiji_api.KijijiApiException: Could not post ad.
It could be a false error where it raises an Exception when it should not. But, I can confirm that @JohnBakery 's solution is working and we could build upon his or her solution. So we are getting somewhere! ^_^
Change if "deleteWithoutSurvey" not in resp.text:
to if "deleteSurveyReasons" not in resp.text:
Since sessions seem to stay alive forever, maybe we can take a valid logged in session, steal the cookies and dump them into Kijiji-Repost-Headless?
That is a good idea. I just hardcoded session id cookie into the code and it's posting ads with no issue. If anyone wants to do the same thing, here is what I did.
kijiji_api.py
1. Add a new line after line #98 and type """ 2. Add a new line after line #118 and type """ 3. After line #119 add the following code
cookie_dict = {'ssid': 'REPLACE_WITH_YOUR_SESSION_ID'}
requests.utils.add_dict_to_cookiejar(self.session.cookies,cookie_dict)
To get your session id.1. Go to kijiji.ca 2. Login into your account if you are not logged in. 3. Press F12 4. In FireFox: Go to Storage. In Chrome: Go to Application. 5. Open up Cookies, select https://www.kijiji.ca, look for a cookie "ssid", double click on the value field and copy paste it into the code where it says REPLACE_WITH_YOUR_SESSION_ID.
Save
kijiji_api.py
, run it.To be clear, do I comment out the lines starting with: login_url ... and ending with: resp = ...
I did the above but some ads post properly and some have failures with image uploads. Thanks for the help.
Yes, that's correct. Not sure about the images. I reposted like 6 ads and none of them had the image issue
@JohnBakery any chance you can make a pull request and have this merged back to master branch so this can be closed? I think we can add the instructions on the readme to provide the session_id as well as one of the parameters passed?
@JohnBakery, I have used the same scripts with the same images to post ads on kijiji.ca for a half year. It's not because of the image sizes for sure. Thank you so much for the helps
Also got the same issue, the ad gets uploaded, but the success check that is based on the presence of "deleteSurveyReasons"
in the resp.text
is not functionning. So the main.py triggers another attempt eventhough the first was successfull. I ended up with the intended ad posted 5 times.
I tried to add this line
print(resp.text)
just to check if the response was ok and it is. It does contain the "deleteSurveyReasons"
Eventhough i am an experienced programmer, i am new to python !
Still getting errors on login with ads not being posted either:
Traceback (most recent call last):
File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/jmshong/Downloads/Kijiji-Repost-Headless-master/kijiji_repost_headless/main.py", line 209, in
I believe that this issue should be fixed now. Please comment if you experience further issue
I am running the following command:
python3 kijiji_repost_headless -u $USER -p $PASS -s $SSID show
But I keep getting the following error:
usage: kijiji_repost_headless [-h] [-s SSID] {post,show,delete,nuke,check_ad,repost,build_ad} ...
kijiji_repost_headless: error: invalid choice: 'xxxxxxxxxx@gmail.com' (choose from 'post', 'show', 'delete', 'nuke', 'check_ad', 'repost', 'build_ad')
Last time when it used to work I would only use this command:
python3 kijiji_repost_headless -u $USER -p $PASS show
I have updated the code to the last from your repo.
What am I doing wrong here? 😢
I am running the following command:
python3 kijiji_repost_headless -u $USER -p $PASS -s $SSID show
But I keep getting the following error:usage: kijiji_repost_headless [-h] [-s SSID] {post,show,delete,nuke,check_ad,repost,build_ad} ... kijiji_repost_headless: error: invalid choice: 'xxxxxxxxxx@gmail.com' (choose from 'post', 'show', 'delete', 'nuke', 'check_ad', 'repost', 'build_ad')
Program arguments -u
and -p
have been removed.
@jackm Thanks! It's still there in the ReadMe so I didn't know. What's the correct command now?
I am using:
python3 kijiji_repost_headless -s $SSID show --key id
but that doesn't work either. I get the following error now:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "kijiji_repost_headless/__main__.py", line 198, in <module> main() File "kijiji_repost_headless/__main__.py", line 50, in main args.function(args) File "kijiji_repost_headless/__main__.py", line 101, in show_ads api.login(args.ssid) File "kijiji_repost_headless/kijiji_api.py", line 102, in login with open(ssid_path) as ssidFile: FileNotFoundError: [Errno 2] No such file or directory: '/Users/xxx/Code/Kijiji-Repost-Headless/xxxxxxxxx'
NEVER MIND: I found the instructions in the updated ReadMe:
Before posting an ad, you will need to manually get your Kijiji SSID.
Use the command like this:
python3 kijiji_repost_headless -s ssid.txt show --key id
@jackm Thanks! It's still there in the ReadMe so I didn't know. What's the correct command now? I am using:
python3 kijiji_repost_headless -s $SSID show --key id
but that doesn't work either. I get the following error now:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "kijiji_repost_headless/__main__.py", line 198, in <module> main() File "kijiji_repost_headless/__main__.py", line 50, in main args.function(args) File "kijiji_repost_headless/__main__.py", line 101, in show_ads api.login(args.ssid) File "kijiji_repost_headless/kijiji_api.py", line 102, in login with open(ssid_path) as ssidFile: FileNotFoundError: [Errno 2] No such file or directory: '/Users/xxx/Code/Kijiji-Repost-Headless/xxxxxxxxx'
NEVER MIND: I found the instructions in the updated ReadMe:
Getting your SSID
Before posting an ad, you will need to manually get your Kijiji SSID.
1. Log into Kijiji in your browser. Make sure "remember me" is checked. 2. Inspect the cookies that have been set by Kijiji for your browser. 3. Find the cookie named "ssid" and copy its value. 4. Paste the value into the "ssid.txt" file.
Use the command like this:
python3 kijiji_repost_headless -s ssid.txt show --key id
May I ask why you're using a different command not in the readme? The commands seem to be the same as before, only difference is to just get your ssid and paste it in ssid.txt. Am I missing something? I'd like to be clear on this before attempting to repost an ad.
@Rings4 It's working now. Thanks
I got my SSID using chrome developer tools, and I'm using
python kijiji_repost_headless -s ssid.txt show repost !dir!\item.yml
I know the path is 100% correct. Keeps giving me "Could not log in"
Also what does " show --key id" do in the parameters?
It worked flawlessly until Oct. 6, but suddenly collapsed with the following error
{"errors":[{"message":"Something went wrong during authentication.","locations":[{"line":2,"column":3}],"path":["loginUser"],"extensions":{"errorCode":"LOGIN_RECAPTCHA_FAIL","statusCode":401}}],"data":{"loginUser":null}}
I'm using the latest master, please help