salesforce-marketingcloud / FuelSDK-Python

FuelSDK for python
Other
127 stars 191 forks source link

[Enhancement] Modify the write location to /tmp/ directory so that fuelsdk can be utilized in aws lambda #128

Open cheneyshreve opened 3 years ago

cheneyshreve commented 3 years ago

Is your feature request related to a problem? Please describe A clear and concise description of what the problem is. I deployed a lambda function that uses fuelsdk-python to authenticate with salesforce marketing cloud api, retrieve data from a data extension, and peform pagination. This works brilliantly when running locally using pure python. When I deploy the logic within an aws python function, using the serverless framework, I get the error below when I execute the function. As a side note, I could never get this to work locally using environment variables, but it always works when I use phyton.config so I deploy to a private aws bucket that does not permit public access. The lamba function itself, however, does have adequate permissions to read/write to the relevant s3 buckets. I believe the error below is down to the fact that aws lambda will only let you write to '/tmp/' directory, so it might be just a very quick code update to do this.

  "errorMessage": "[Errno 30] Read-only file system: '/var/task/FuelSDK/ExactTargetWSDL.xml'",
  "errorType": "OSError",
  "stackTrace": [
    "  File \"/var/task/handler.py\", line 112, in paginate\n    myClient = ET_Client.ET_Client()\n",
    "  File \"/var/task/FuelSDK/client.py\", line 56, in __init__\n    self.configure_client(get_server_wsdl, params, tokenResponse)\n",
    "  File \"/var/task/FuelSDK/client.py\", line 130, in configure_client\n    self.wsdl_file_url = self.load_wsdl(wsdl_server_url, wsdl_file_local_location, get_server_wsdl)\n",
    "  File \"/var/task/FuelSDK/client.py\", line 224, in load_wsdl\n    self.retrieve_server_wsdl(wsdl_url, file_location)\n",
    "  File \"/var/task/FuelSDK/client.py\", line 241, in retrieve_server_wsdl\n    f = open(file_location, 'w')\n"
  ]
}

I think the resolution, for aws lambda, would just be to modify line 241 of client.py

f = open(file_location, 'w')

modification would likely need to be this, because this is similar to how you write to s3 buckets via lambda: f = open('/tmp/'+ file_location, 'w')

I'm uncertain, if I were to clone this repo and make the modification locally for reference, how I would then reference that via the traditional requirements route (e.g. using pip and requirements.txt that are utilized by aws lambda).

Describe the solution you'd like A clear and concise description of what you want to happen. to be able to use fuelsdk-python within aws lambda function

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. deploying with amplify or another route on aws. lambda has much greater cost benefits.

Additional context Add any other context or screenshots about the feature request here. I'd be happy to contribute if someone can point me in the right direction. The fuelsdk is really valuable to the work that I'm doing and it would be amazing to be able to use it with aws lambda.

vijayprakashpj commented 3 years ago

The WSDL file will be downloaded to the location of your choice if you set wsdl_file_local_loc in params and get_server_wsdl in the constructor argument.

Refer this: https://github.com/salesforce-marketingcloud/FuelSDK-Python/blob/master/FuelSDK/client.py#L223

Example:

stubObj = ET_Client.ET_Client(
  True, False,
  {
      'clientid': '<CLIENT_ID>',
      'clientsecret': '<CLIENT_SECRET>',
      'defaultwsdl': 'https://webservice.exacttarget.com/etframework.wsdl',
      'authenticationurl': '<AUTH TENANT SPECIFIC ENDPOINT>',
      'baseapiurl': '<REST TENANT SPECIFIC ENDPOINT>',
      'soapendpoint': '<SOAP TENANT SPECIFIC ENDPOINT>',
      'wsdl_file_local_loc': r'/tmp/ExactTargetWSDL.xml',
      'useOAuth2Authentication': 'True',
      'accountId': '<TARGET_ACCOUNT_ID>',
      'scope': '<PERMISSION_LIST>'
      'applicationType': '<APPLICATION_TYPE>'
      'redirectURI': '<REDIRECT_URI_FOR_PUBLIC/WEB_APP>'
      'authorizationCode': '<AUTHORIZATION_CODE_FOR_PUBLIC/WEB_APP>'
  })
cheneyshreve commented 3 years ago

Hi Vijay,

Many thanks for your quick response! Per your comment; I have been trying to get the function to read the wsdl file from an s3 bucket without success. I was hoping it was possible to use the default option (e.g. omission) with /tmp/ update as I'd mentioned, but hopefully I'll be able to sort out why it's not reading from the s3 bucket. Thanks again for your response, which I very much appreciate.

Best, Cheney

On Tue, Jan 12, 2021 at 1:08 PM Vijay Prakash P J notifications@github.com wrote:

The WSDL file will be downloaded to the location of your choice if you set wsdl_file_local_loc in params and get_server_wsdl in the constructor argument.

Refer this: https://github.com/salesforce-marketingcloud/FuelSDK-Python/blob/master/FuelSDK/client.py#L223

Example:

stubObj = ET_Client.ET_Client( True, False, { 'clientid': '', 'clientsecret': '', 'defaultwsdl': 'https://webservice.exacttarget.com/etframework.wsdl', 'authenticationurl': '', 'baseapiurl': '', 'soapendpoint': '', 'wsdl_file_local_loc': r'/tmp/ExactTargetWSDL.xml', 'useOAuth2Authentication': 'True', 'accountId': '', 'scope': '' 'applicationType': '' 'redirectURI': '<REDIRECT_URI_FOR_PUBLIC/WEB_APP>' 'authorizationCode': '<AUTHORIZATION_CODE_FOR_PUBLIC/WEB_APP>' })

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/salesforce-marketingcloud/FuelSDK-Python/issues/128#issuecomment-758907352, or unsubscribe https://github.com/notifications/unsubscribe-auth/APC7ZFZHQFTA4BVMQ3ZJYZLSZST4FANCNFSM4V7RSAEA .

-- Cheney Shreve, Ph.D. Pronouns: she, her, hers Senior Software Engineer, Enterprise CRM 434.566.3155

ulinares commented 3 years ago

Hi, an unrelated comment to the issue, @cheneyshreve how do you perform pagination on the data extension? Thanks.