Closed modean closed 3 years ago
Thank you for your report. Could you provide more details about your code and environment? I can use 1.10.0 and a simple script to create, list, get, put, and delete objects and buckets without seeing this exception.
@IBMeric basically we instantiate the SDK like this:
import IBMS3 from "ibm-cos-sdk/clients/s3"
new IBMS3({
endpoint: endpoint,
serviceInstanceId: storageInstanceId,
sslEnabled: true,
s3ForcePathStyle: true,
// @ts-ignore
tokenManager: {
getToken: function () {
return { accessToken: token };
}
}
});
On the instance we then call upload
with the following upload options:
uploadOptions = {
Bucket: auth.bucket,
Key: key,
Body: body,
progressEventListener: /* some function */
};
IBMS3.upload(uploadOptions, function (err, dataObj) {}
Our environment is a React microservice that runs on a Node 14.16.0 runtime. The code from above is part of the client-side code of the UI microservice.
Not sure whether this helps answering your question. I am also available on IBM Slack, feel free to reach out.
@IBMeric - any updates? I've hit the same issue while upgrading from "ibm-cos-sdk": "1.9.0"
-> "ibm-cos-sdk": "1.10.0"
Some changes in this diff must cause the problem
Our code fails in this line (running in a browser environment) because AWS.util.isBrowser is not a function https://github.com/IBM/ibm-cos-sdk-js/blob/b86487e106df1d4d4d545384c2c20ed6c4e02001/lib/http.js#L115
The stack trace that lead to the invocation of this line is below
# the exact line numbers I have access to wouldn't help because they refer to bundled code on our side
at HttpRequest.getUserAgentHeaderName
at HttpRequest.setUserAgent
at new HttpRequest
at new Request
at features.constructor.makeRequest
at features.constructor.svc.<computed> [as putObject]
at ManagedUpload.nextChunk
at ManagedUpload.fillBuffer
at ManagedUpload.send
Below lines were removed in the referenced diff and at least look suspicious: https://github.com/IBM/ibm-cos-sdk-js/blob/9bec9207a1df0404d3cfeef42234cdea501a5253/lib/util.js#L38-L39
Is there anything else you need to drive this forward?
This seems to be an interaction issue with the way the SDK is being used. When using require('ibm-cos-sdk')
, everything works as expected. When only importing from a directory in the SDK, the above exception can happen. The workaround is to add those two lines above back in. This issue has been reproduced and is being tracked internally.
For reference: https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-node#node-examples-init
I created a new project from scratch for test purposes and am unable to reproduce the exception (the previous reproduction turned out to be an unrelated problem). Could you provide additional details on differences between this sample code and your project? Since I cannot reproduce the problem in pure Javascript, are you using a dialect? Which one?
npm install ibm-cos-sdk
main.js
import S3 from 'ibm-cos-sdk/clients/s3.js'
import myCreds from './creds.js'
const s3Config = {
endpoint: 'https://' + myCreds.api_endpoint + ':8443',
apiKeyId: myCreds.ibm_api_key,
serviceInstanceId: myCreds.ibm_service_instance_id,
ibmAuthEndpoint: myCreds.ibm_auth_endpoint,
signatureVersion: 'iam',
// accessKeyId: myCreds.aws_access_key_id,
// secretAccessKey: myCreds.aws_secret_access_key,
// signatureVersion: 'v4',
s3ForcePathStyle: true,
};
var s3Client = new S3(s3Config);
async function main() {
var bucketName = `cos-${Date.now()}`;
console.log('createBucket');
var createBucketParams = {Bucket: bucketName, CreateBucketConfiguration: {LocationConstraint: ''}};
var response = await s3Client.createBucket(createBucketParams).promise();
console.log('deleteBucket');
response = await s3Client.deleteBucket({Bucket: bucketName}).promise();
console.log('Complete!');
}
main();
package.json
{
"type": "module"
}
@IBMeric the most obvious difference from your test code and the snippet shared by @modean is that you tested with s3Client.createBucket
, s3Client.deleteBucket
but the problem surfaces when s3Client.upload
is used.
I can try to work on a small reproducer next week but first I'd like to understand if you were able to run an upload and could verify that the code successfully passed this line https://github.com/IBM/ibm-cos-sdk-js/blob/b86487e106df1d4d4d545384c2c20ed6c4e02001/lib/http.js#L115
If so, can you add a breakpoint in there and share what properties are set on AWS.util
?
@daka1510 Perhaps see #83. There is no difference when I use s3Client.upload
.
import stream from 'stream';
import fs from 'fs';
...
console.log('upload');
var key = 'foo';
var body = 'bar';
var filename = 'mydata.txt';
var readStream = fs.createReadStream(filename);
console.log(readStream);
var params = {Bucket: bucketName, Key: key, Body: readStream};
var options = {partSize: 10 * 2**20, queueSize: 10};
response = await s3Client.upload(params, options).promise();
console.log('delete');
response = await s3Client.deleteObject({Bucket: bucketName, Key: key}).promise();
I added a breakpoint at the line you requested. I truncated multiline output (it's over 200 lines), but you will see the two functions of interest are there.
> console.log(AWS.util)
undefined
< {
< environment: 'nodejs',
< engine: [Function: engine],
< userAgent: [Function: userAgent],
< uriEscape: [Function: uriEscape],
< uriEscapePath: [Function: uriEscapePath],
< urlParse: [Function: urlParse],
< urlFormat: [Function: urlFormat],
< queryStringParse: [Function: queryStringParse],
< queryParamsToString: [Function: queryParamsToString],
< readFileSync: [Function: readFileSync],
< base64: { encode: [Function: encode64], decode: [Function: decode64] },
< buffer: { <truncated> },
< string: { <truncated> },
< ini: { parse: [Function: string] },
< fn: { <truncated> },
< date: { <truncated> },
< crypto: { <truncated> },
< abort: {},
< each: [Function: each],
< arrayEach: [Function: arrayEach],
< update: [Function: update],
< merge: [Function: merge],
< copy: [Function: copy],
< isEmpty: [Function: isEmpty],
< arraySliceFn: [Function: arraySliceFn],
< isType: [Function: isType],
< typeName: [Function: typeName],
< error: [Function: error],
< inherit: [Function: inherit],
< mixin: [Function: mixin],
< hideProperties: [Function: hideProperties],
< property: [Function: property],
< memoizedProperty: [Function: memoizedProperty],
< hoistPayloadMember: [Function: hoistPayloadMember],
< computeSha256: [Function: computeSha256],
< computeContentMD5: [Function: computeContentMD5],
< isClockSkewed: [Function: isClockSkewed],
< applyClockOffset: [Function: applyClockOffset],
< extractRequestId: [Function: extractRequestId],
< addPromises: [Function: addPromises],
< promisifyMethod: [Function: promisifyMethod],
< isDualstackAvailable: [Function: isDualstackAvailable],
< calculateRetryDelay: [Function: calculateRetryDelay],
< handleRequestWithRetries: [Function: handleRequestWithRetries],
< uuid: { v4: [Function: uuidV4] },
< convertPayloadToString: [Function: convertPayloadToString],
< defer: [Function: defer],
< getRequestPayloadShape: [Function: getRequestPayloadShape],
< getProfilesFromSharedConfig: [Function: getProfilesFromSharedConfig],
< defaultProfile: 'default',
< configOptInEnv: 'AWS_SDK_LOAD_CONFIG',
< sharedCredentialsFileEnv: 'AWS_SHARED_CREDENTIALS_FILE',
< sharedConfigFileEnv: 'AWS_CONFIG_FILE',
< imdsDisabledEnv: 'AWS_EC2_METADATA_DISABLED',
**< isBrowser: [Function (anonymous)],**
**< isNode: [Function (anonymous)],**
< Buffer: [Function: Buffer] { <truncated> },
< domain: { <truncated> },
< stream: <ref *1> [Function: Stream] { <truncated> },
< url: { <truncated> },
< querystring: { <truncated> },
< iniLoader: IniLoader { resolvedProfiles: {} }
< }
@IBMeric - I created a small repository with code that allows to reproduce the problem in isolation. See https://github.com/daka1510/ibm-cos-sdk-issue-reproducer#reproducer-for-httpsgithubcomibmibm-cos-sdk-jsissues84 for details.
Most of the code is generated by https://reactjs.org/docs/create-a-new-react-app.html#create-react-app. The second commit adds what is needed to reproduce the problem.
Can you give it a try?
Thank you for the project. I have been able to reproduce the issue in Firefox 88. The problem appears to be an interaction with React that could also apply to other frameworks. I need to discuss this internally and have opened a tracking ticket.
@rrodriguez06 isn't this exactly the line that was removed and thus did cause the breakage?
@IBMeric @modean
For those who are still experiencing this issue, here is an hotfix of the problem.
const AWS = require('ibm-cos-sdk')
AWS.util.isBrowser = function isBrowser() { return process && process.browser; };
AWS.util.isNode = function isNode() { return !AWS.util.isBrowser(); };
it's a bit tricky !
Sorry for a delayed response. I've tested a resolution against @daka1510's project. I no longer see output in the browser console. The plan is to publish this fix in the next point release. Currently there are some other fixes we are working on. If those appear to require too much time, we'll do a separate update for this fix.
@IBMeric - can you share a rough release timeline for this fix?
The release will either be this week or early next week.
@modean We have published 1.10.1, and I have verified that @daka1510's example project prints no errors to the console. Could you verify your issue is resolved and then close this ticket?
Closing this issue as resolved.
@IBMeric I verified the published version 1.10.1 locally and it seems to be working.
Since
"ibm-cos-sdk": "1.10.0",
theisBrowser
function seems to be missing inmaster/lib/util.js
. Thus the following exception will be thrown if using the SDK.