Logon failed,

[shaohui-liu2000]

I called this line first,
self.bulk = SalesforceBulk(username=self.username, password=self.password)

After passing username and password, I got this error below. Is salesforce_oauth_request still alive?

_File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/salesforce_oauth_request/utils.py", line 144, in oauth_flow assert m is not None, "Couldn't find location.href expression in page %s (Username or password is wrong)" % r2.url AssertionError: Couldn't find location.href expression in page https://c.gus.content.force.com/secur/contentDoor?startURL=%2Fsetup%2Fsecur%2FRemoteAccessAuthorizationPage.apexp%3Fsource%3DCAAAAVSDPPi2ME8wQTAwMDAwMDAwMDA0AAAAyMwstB_dCWYFjhgJ0sb1UgI69tUv9aPyKZIonyGrB1a-Dnq75JGtPs95o2bOpwh44EU8iN46Ty7nNJwxK57Y2lHC1FJti3wadR4X8fJfGH7zXmFXL5YenOZgKcXxGJE9ghd9D_DPC-wwW-WVW7b6w7ozDQqV2mDRYd2nXjYkdXPxxLy0uPVdXd-5YCnKMpCbQOlONkdkQApYeWpg7lFeZIYeGYlBzP2LhpvxfRwAwTgPOX_WY2jL1ldvv1gTyXJsq-cpJwCtSM3Pw1aDMwUPuT2UVyxcy2gFkWlaAtf76cc57MYobkLxnUpOl0vKsdTOu8cHsc9WkhRDesN8JzCVykTWalW4V7_KABVwXTbzi0VQNc-Hyf8M9SXiNDTtFmMaO3tP22EC4XQBNORXyzkliWxSHV281EL39q5TVYnx6f5TrOWwoKyIYeg2ldPx4oaOXfTbYcWqDBiQHPTp2I5Ia0E%253D%26display%3Dpopup&sid=00DB000000043rg%21AQYAQHf9AvvMNHl8FbKP5fZadEkwlpjaimPcP731omS_e760IuwyrjUfvwnKekWzTr1dzxuhJjc0edfKAvwC7AOmDk5lC03i&skipRedirect=1&lm=CAAAAVSC0PYUME8wQjAwMDAwMDAwMDAxAAAAyuQrAfGJ0xESGClbUbGQpO4m4KNDTizP0Ooy-SZrj5g4HmETo_2FnXK7m64RM0eXxM8E_rgHjkedEEkZ9yeF2RDzLs43WcqsBwGDcWwwxDn1 (Username or password is wrong)_

[lambacck]

I'm not sure about the status of salesforce_oauth_request. Given that it scrapes the salesforce login page I'd guess that it could be easily broken.

I've successfully used salesforce-oauth2 recently but you then need an oauth flow for that to work.

I have some example code of getting the oauth authorization using a web flow and then using the oauth token with bulk.

[shaohui-liu2000]

After combining with beatbox for login session id, this lib worked great!

[jmagana2000]

This worked for me. Did not show step where I get credentials from a yaml file(cfg). The client_id, client_secret, and callback come from the API (Enable OAuth Settings) in Salesforce:

def authorise(cfg): consumer_key = cfg['salesforce']['client_id'] consumer_secret = cfg['salesforce']['client_secret'] username = cfg['salesforce']['user'] password = cfg['salesforce']['passwd'] + cfg['salesforce']['token'] login_server = 'https://login.salesforce.com' token_url = login_server+'/services/oauth2/token' params = urllib.urlencode({ 'grant_type': 'password', 'client_id': consumer_key, 'client_secret': consumer_secret, 'username': username, 'password': password }) data = urllib2.urlopen(token_url, params).read() oauth = json.loads(data) return oauth

def main(): os.environ["SALESFORCE_CLIENT_ID"]=cfg['salesforce']['client_id'] os.environ["SALESFORCE_CLIENT_SECRET"]=cfg['salesforce']['client_secret'] os.environ["SALESFORCE_REDIRECT_URI"]=cfg['salesforce']['redirect_uri'] oauth = authorise(cfg) token = oauth['access_token'] bulk = SalesforceBulk(sessionId=token, host=urlparse(cfg['salesforce']['host']).hostname)

[bholagabbar]

I figured out a way to fix this. We can login using simple-salesforce, fetch the session id and host from there and then use this to login through salesforce-bulk. Here is the complete code example:

from salesforce_bulk import SalesforceBulk
from salesforce_bulk import CsvDictsAdapter
from simple_salesforce import Salesforce
import ConfigParser

config = ConfigParser.RawConfigParser()
config.read('config.properties')

username = config.get('Section', 'username')
password = config.get('Section', 'password')
client_id = config.get('Section', 'client_id')
client_secret = config.get('Section', 'client_secret')
redirect_uri = config.get('Section', 'redirect_uri')

security_token = config.get('Section', 'security_token')

# Authenticate
sf = Salesforce(username=username, password=password, security_token=security_token)
bulk = SalesforceBulk(sessionId=sf.session_id, host=sf.sf_instance)

# Done! Make a bulk API request

[EamonnONeill]

@bholagabbar thank you!!

[bholagabbar]

Screw this library, it's all broken. I've fixed all the crucial issues and released a better library https://github.com/wingify/salesforce-bulkipy

[bholagabbar]

^ It's well documented and available on pip as well, take a look

[lambacck]

@bholagabbar The password authentication as exists in this library is a giant hack because you need an OAuth session token. salesforce_oauth_request.login provides that right now but it's not reliable. simple-salesforce provides you with that. I'd be willing to entertain patches that replace salesforce_oauth_request.login with the simple_salesforce equivalent. The big hold up to merging things is that the tests are neither automated, nor extensive. simple_salesforce is quickly becoming the way to salesforce API rest calls and I'd support making this library a wrapper around simple_salesforce in general that provides a nice api for bulk.

[bholagabbar]

@lambacck I've fixed the login in my library using simple-salesforce.

class SalesforceBulkipy(object):
    def __init__(self, session_id=None, host=None, username=None, password=None, security_token=None, sandbox=False,
                 exception_class=BulkApiError, API_version="29.0"):
        if (not session_id or not host) and (not username or not password or not security_token):
            raise RuntimeError(
                "Must supply either sessionId,host or username,password,security_token")
        if username and password and security_token:
            session_id, host = SalesforceBulkipy.login_to_salesforce_using_username_password(username, password,
                                                                                             security_token, sandbox)

        if host[0:4] == 'http':
            self.endpoint = host
        else:
            self.endpoint = "https://" + host
        self.endpoint += "/services/async/%s" % API_version
        self.sessionId = session_id
        self.jobNS = 'http://www.force.com/2009/06/asyncapi/dataload'
        self.jobs = {}  # dict of job_id => job_id
        self.batches = {}  # dict of batch_id => job_id
        self.batch_statuses = {}
        self.exception_class = exception_class

    @staticmethod
    def login_to_salesforce_using_username_password(username, password, security_token, sandbox):
        sf = simple_salesforce.Salesforce(username=username, password=password, security_token=security_token,
                                          sandbox=sandbox)
        return sf.session_id, sf.sf_instance

[bholagabbar]

Do you want a clean PR? @lambacck

[bholagabbar]

My library is tried and tested, I see you've merged every change mentioned in my library except the login

[bholagabbar]

cc @lambacck

[lambacck]

I would be happy to review a clean PR for the password based auth that worked against simple-salesforce instead of the totally unmaintained library we're using now that doesn't really work.

[tickylum]

@lambacck @bholagabbar Dears, I'm still not able to use salesforce_bulkipy as I'm always getting "Invalid session id" [400] error. Can you please guide me what am I doing wrong?

[bholagabbar]

@tickylum SalesforceBulkipy is an independent library forked from salesforce-bulk. It was released by me and used in production system @wingify.

You can install it using pip install salesforce-bulkipy and you can view it on GitHub here https://github.com/wingify/salesforce-bulkipy

You should find relevant coding snippet examples in the README.

Not exactly sure what problem you are encountering.

[tickylum]

Hello Shreyans,

Thanks a lot for your response.

I'm able to authenticate to Salesforce, but this line is throwing an exception

job = bulk.create_query_job("Object_Name", contentType='CSV')

saying the following:

salesforce_bulkipy.salesforce_bulkipy.BulkApiError: [400] Bulk API HTTP Error result: <?xml version="1.0" encoding="UTF-8"?><error

xmlns="http://www.force.com/2009/06/asyncapi/dataload">

InvalidSessionId Invalid session id

Regards, Mina Michel

On Thu, Mar 16, 2017 at 12:39 PM, Shreyans Sheth notifications@github.com wrote:

@tickylum https://github.com/tickylum SalesforceBulkipy is an independent library forked from salesforce-bulk. It was released by me and used in production system @wingify https://github.com/wingify.

You can install it using pip install salesforce-bulkipy and you can view it on GitHub here https://github.com/wingify/salesforce-bulkipy

You should find relevant coding snippet examples in the README.

Not exactly sure what problem you are encountering.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-287019190, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4maiv_fekPVkg6-KNvlMYxGUt4A7fJks5rmRF-gaJpZM4IYZt_ .

[bholagabbar]

@tickylum I remember encountering that problem when I used the same session for over 5 hours. When we deployed the system, I would initiate the job every 30 minutes. However, after 5 hours it would throw me the exact same error as the session had expired. The workaround is to newly authenticate (basically authenticate again) for a new bulk job.

[tickylum]

This issue is happening from the beginning and always :( I'm guessing it has to do with the packages I'm using... Their versions maybe... What do you think?

Regards, Mina Michel

On Thu, Mar 16, 2017 at 12:51 PM, Shreyans Sheth notifications@github.com wrote:

@tickylum https://github.com/tickylum I remember encountering that problem when I used the same session for over 5 hours. When we deployed the system, I would initiate the job every 30 minutes. However, after 5 hours it would throw me the exact same error as the session had expired. The workaround is to newly authenticate (basically authenticate again) for a new bulk job.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-287021961, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4mamjbL8WmjMQqhU4kdTCB0iRK2acjks5rmRQ6gaJpZM4IYZt_ .

[bholagabbar]

I have no idea...we are currently using the library successfully in production. I was an intern there but AFAIK we are still using it without any problems. Maybe it has something to do with your session timeout settings

[lambacck]

@tickylum there is insufficient information in your message to tell what is going on. Where is your sessionId coming from? Have you set the correct hostname? Are you authenticating against production or a sandbox?

[lambacck]

@tickylum I would the simple-salesforce example from this thread to work and probably other attempts to use username and password directly with this library to not work because the dependency that provides username/password to that this package currently depends on isn't reliable.

[tickylum]

I'm using the same parameters of my code... Into workbench and it's working fine there, so it must be something wrong with the code. The issue arises with production and sandbox. Haven't set the hostname because it resolves alone...

[tickylum]

Simple Salesforce worked from the first time but it doesn't support bulk operations right?

On Mar 16, 2017 5:00 PM, "Christopher Lambacher" notifications@github.com wrote:

@tickylum https://github.com/tickylum I would the simple-salesforce example from this thread to work and probably other attempts to use username and password directly with this library to not work because the dependency that provides username/password to that this package currently depends on isn't reliable.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-287084583, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4maqW1g1bEv-wBlYYN0BRCmxikqphpks5rmU5-gaJpZM4IYZt_ .

[lambacck]

@tickylum SimpleSalesforce will give you an OAuth token (aka sessionId) that you can use with this library if you want to use username and password. Username and password to this library won't work.

[lambacck]

@tickylum See this comment: https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-265721078

[bholagabbar]

@lambacck My library Salesforce Bulkipy routes it's login through simple-salesforce itself.

[lambacck]

@bholagabbar yes, I'm aware of that. As I stated in earlier in this thread I would be happy to review and accept a clean PR that made that update. I'd very much like to not fork this library.

[tickylum]

@lambacck, I see... Now I think I remember I used this method long ago... But just recalled it when you mentioned it. @bholagabbar, so yes I thought your library would use both other packages and by pass this session issue but it didn't with me :( I've followed the manual as is. Authentication succeeded, but couldn't create a job afterwards (invalid session id)

[bholagabbar]

@lambacck I would have but I was on a tight timeline back on my internship then and a lot of the PRs hadn't been merged. Plus, had you even updated the github, updating the library on Python Package Index (pip) was also required. I just decided to take things into my own hands, fork it, make the changes and publish. I''m glad you're maintaining the library now :)

[tickylum]

@bholagabbar, okay, so I have some news here. Looks to me that it's an org issue. The same code is able to "create" the job on a developer org, but not on a production org. Are there any settings you are aware of it that I should do on the org to make the code work?

[bholagabbar]

Not sure if you're referring to 'developer org' as the salesforce sandbox. You can add an extra parameter called 'sandbox=True' while intialiazling the library. See the code examples..By default it is on production. You may be generating incorrect keys for your developer or sandbox accounts.. Something like that.

On 18-Mar-2017 5:43 pm, "Mina Michel" notifications@github.com wrote:

@bholagabbar https://github.com/bholagabbar, okay, so I have some news here. Looks to me that it's an org issue. The same code is able to "create" the job on a developer org, but not on a production org. Are there any settings you are aware of it that I should do on the org to make the code work?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-287541283, or mute the thread https://github.com/notifications/unsubscribe-auth/ALJuG9Q2-unjiI7UgNWR9gc_bdKabenMks5rm8plgaJpZM4IYZt_ .

[tickylum]

@bholagabbar, no... I meant "developer org" and not a sandbox. Yes I know about the parameter and so, it's not working on production or sandbox, but it's working on another totally different dev org... This means there is a specific setting that should be done on the production, right? Any ideas?

[bholagabbar]

Still not sure what you mean by developer and production orgs. I guess they're two entirely separate organizations. Since the library works on one of them, it is safe to say that the library is functioning as intended. You can cross check and keep the settings same across the 2 different orgs.

On 18-Mar-2017 11:14 pm, "Mina Michel" notifications@github.com wrote:

@bholagabbar https://github.com/bholagabbar, no... I meant "developer org" and not a sandbox. Yes I know about the parameter and so, it's not working on production or sandbox, but it's working on another totally different dev org... This means there is a specific setting that should be done on the production, right? Any ideas?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-287562167, or mute the thread https://github.com/notifications/unsubscribe-auth/ALJuG8uM9BIhs-f3XcE9LDv9hPpS26nAks5rnBftgaJpZM4IYZt_ .

[bholagabbar]

Are you sure you are having admin credentials on the org it is not working?

On 18-Mar-2017 11:14 pm, "Mina Michel" notifications@github.com wrote:

@bholagabbar https://github.com/bholagabbar, no... I meant "developer org" and not a sandbox. Yes I know about the parameter and so, it's not working on production or sandbox, but it's working on another totally different dev org... This means there is a specific setting that should be done on the production, right? Any ideas?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-287562167, or mute the thread https://github.com/notifications/unsubscribe-auth/ALJuG8uM9BIhs-f3XcE9LDv9hPpS26nAks5rnBftgaJpZM4IYZt_ .

[tickylum]

Yes, I'm a system administrator on all the orgs I'm trying on. It worked with my Developer Org, but not on the Enterprise Org... There must be a configuration done on the org for the code to run?

Regards, Mina Michel

On Tue, Mar 21, 2017 at 4:58 PM, Shreyans Sheth notifications@github.com wrote:

Are you sure you are having admin credentials on the org it is not working?

On 18-Mar-2017 11:14 pm, "Mina Michel" notifications@github.com wrote:

@bholagabbar https://github.com/bholagabbar, no... I meant "developer org" and not a sandbox. Yes I know about the parameter and so, it's not working on production or sandbox, but it's working on another totally different dev org... This means there is a specific setting that should be done on the production, right? Any ideas?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28# issuecomment-287562167, or mute the thread https://github.com/notifications/unsubscribe-auth/ALJuG8uM9BIhs- f3XcE9LDv9hPpS26nAks5rnBftgaJpZM4IYZt_ .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-288105277, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4maoB4oM_lVv0cdesWzE1bMZF9xDh5ks5rn-V4gaJpZM4IYZt_ .

[lambacck]

Not all editions of Salesforce come with API. Are you sure yours does? https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm

[tickylum]

I've tried 2 orgs (enterprise edition) and they both didn't work resulting with the same error message, but it worked fine from the first time on the developer org. Have you used it before on an enterprise org? If yes, did you do change any settings on the org?

On Wed, Mar 22, 2017 at 3:45 PM, Christopher Lambacher < notifications@github.com> wrote:

Not all editions of Salesforce come with API. Are you sure yours does? https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_ cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_applimits platform_api.htm

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-288402254, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4marHDkD2DyPsWKG-oPfZv3FCPBb6Vks5roSXegaJpZM4IYZt_ .

[lambacck]

@tickylum I have used this library against thousands of salesforce orgs. Assuming I have API access at all, this library works. The only caveat is that I use a real oauth flow and not any kind of password thing. I'd check that API calls work with just simple-salesforce. If that doesn't work you should open a case with Salesforce.

[bholagabbar]

@ticklyum Try using the simple-salesforce library to login and perform a task and see if that works. That should tell whether it's a problem with the org or not

On 23-Mar-2017 7:46 pm, "Christopher Lambacher" notifications@github.com wrote:

@tickylum https://github.com/tickylum I have used this library against thousands of salesforce orgs. Assuming I have API access at all, this library works. The only caveat is that I use a real oauth flow and not any kind of password thing. I'd check that API calls work with just simple-salesforce. If that doesn't work you should open a case with Salesforce.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-288732576, or mute the thread https://github.com/notifications/unsubscribe-auth/ALJuG1ccvxpc-kbo2Gpa7Xnx_IVuWes8ks5ron6zgaJpZM4IYZt_ .

[manusinghal19]

@tickylum I am facing the exact same issue. Have you found the work around for this problem?

[tickylum]

Nope. I did not eventually know why wasn't it working on my Enterprise org, although I was able to get it to work on a dev org. So i'm guessing it has to do with some settings on the Enterprise org?! Please let me know if you reached a solution about this?

Regards, Mina Michel

On Wed, Jun 14, 2017 at 12:52 PM, Manu Singhal notifications@github.com wrote:

@tickylum https://github.com/tickylum I am facing the exact same issue. Have you found the work around for this problem?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/28#issuecomment-308396165, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4makXoKPbNRLXj4401J00TkrxVYKZ8ks5sD7uKgaJpZM4IYZt_ .

[jbhardwaj2101]

@lambacck I've fixed the login in my library using simple-salesforce.

class SalesforceBulkipy(object):
    def __init__(self, session_id=None, host=None, username=None, password=None, security_token=None, sandbox=False,
                 exception_class=BulkApiError, API_version="29.0"):
        if (not session_id or not host) and (not username or not password or not security_token):
            raise RuntimeError(
                "Must supply either sessionId,host or username,password,security_token")
        if username and password and security_token:
            session_id, host = SalesforceBulkipy.login_to_salesforce_using_username_password(username, password,
                                                                                             security_token, sandbox)

        if host[0:4] == 'http':
            self.endpoint = host
        else:
            self.endpoint = "https://" + host
        self.endpoint += "/services/async/%s" % API_version
        self.sessionId = session_id
        self.jobNS = 'http://www.force.com/2009/06/asyncapi/dataload'
        self.jobs = {}  # dict of job_id => job_id
        self.batches = {}  # dict of batch_id => job_id
        self.batch_statuses = {}
        self.exception_class = exception_class

    @staticmethod
    def login_to_salesforce_using_username_password(username, password, security_token, sandbox):
        sf = simple_salesforce.Salesforce(username=username, password=password, security_token=security_token,
                                          sandbox=sandbox)
        return sf.session_id, sf.sf_instance

Hi I am using the SalesforceBulkipy. But I am not able to login with my credentials. I am getting an error: init() got an unexpected keyword argument 'sandbox' However, I am providing sandbox=True. Even if I am not providing it, I should be able to log in. But I am not. Could you please help me?

[lambacck]

@jbhardwaj2101 SalesforceBulkipy is a long abandoned fork of this project. Version 2.2.0 of salesforce-bulk (this library) works with the latest version of simple-salesforce. I am unable to help you with SalesforceBulkipy.

[jbhardwaj2101]

@jbhardwaj2101 SalesforceBulkipy is a long abandoned fork of this project. Version 2.2.0 of salesforce-bulk (this library) works with the latest version of simple-salesforce. I am unable to help you with SalesforceBulkipy.

@lambacck Thank you Christopher! I have tried using the Salesforce-Bulk. I was able to login. But the problem is that I am trying to pull 1.5 million records from salesforce into a pandas dataframe but salesforce-bulk is not able to pull that large volume of data. Is there a workaround?

[jbhardwaj2101]

@bholagabbar Hi I am using the SalesforceBulkipy. But I am not able to login with my credentials. I am getting an error: init() got an unexpected keyword argument 'sandbox' However, I am providing sandbox=True. Even if I am not providing it, I should be able to log in. But I am not. Could you please help me?

[bholagabbar]

@jbhardwaj2101 like Chris said, salesforce-bulkipy hasn't been maintained (guilty!). However, most of it's changes were backported to salesforce-bulk so I recommend you use that as clearly, it's still being maintained.