search

DmitryMezhensky / Hadoop-and-Swift-integration

API to run Hadoop MapReduce programs over Swift
20 stars 15 forks source link

Authentication Broken in HPCloud #3

Open smith1511 opened 11 years ago

smith1511 commented 11 years ago

HPCloud has added a new field to the service catalog endpoints: PublicURL2 which breaks the JSON -> Object mapping in the Swift impl. (see exception below).

Maybe setting '@JsonIgnoreProperties(ignoreUnknown = true)' would be sufficient?

java.lang.RuntimeException: Error generating response at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:76) at org.apache.hadoop.fs.swift.http.SwiftRestClient$10.extractResult(SwiftRestClient.java:1051) at org.apache.hadoop.fs.swift.http.SwiftRestClient$10.extractResult(SwiftRestClient.java:1013) at org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1301) at org.apache.hadoop.fs.swift.http.SwiftRestClient.authenticate(SwiftRestClient.java:1013) at org.apache.hadoop.fs.swift.http.SwiftRestClient.authIfNeeded(SwiftRestClient.java:1230) at org.apache.hadoop.fs.swift.http.SwiftRestClient.preRemoteCommand(SwiftRestClient.java:1247) at org.apache.hadoop.fs.swift.http.SwiftRestClient.headRequest(SwiftRestClient.java:950) at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.getObjectMetadata(SwiftNativeFileSystemStore.java:163) at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem.getFileStatus(SwiftNativeFileSystem.java:155) at org.apache.hadoop.fs.FileSystem.getFileStatus(FileSystem.java:1388) at org.apache.hadoop.fs.FileSystem.globStatusInternal(FileSystem.java:1057) at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:998) at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:976) at org.apache.hadoop.tools.distcp2.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77) at org.apache.hadoop.tools.distcp2.CopyListing.buildListing(CopyListing.java:81) at org.apache.hadoop.tools.distcp2.DistCp.createInputFileListing(DistCp.java:325) at org.apache.hadoop.tools.distcp2.DistCp.execute(DistCp.java:146) at org.apache.hadoop.tools.distcp2.DistCp.run(DistCp.java:114) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.tools.distcp2.DistCp.main(DistCp.java:371) Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "publicURL2" (Class org.apache.hadoop.fs.swift.auth.entities.Endpoint), not marked as ignorable at [Source: java.io.StringReader@7a9300cc; line: 76, column: 24](through reference chain: org.apache.hadoop.fs.swift.auth.AuthenticationWrapper["access"]->org.apache.hadoop.fs.swift.auth.AuthenticationResponse["serviceCatalog"]->org.apache.hadoop.fs.swift.auth.entities.Catalog["endpoints"]->org.apache.hadoop.fs.swift.auth.entities.Endpoint["publicURL2"]) at org.codehaus.jackson.map.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:53) at org.codehaus.jackson.map.deser.StdDeserializationContext.unknownFieldException(StdDeserializationContext.java:246) at org.codehaus.jackson.map.deser.StdDeserializer.reportUnknownProperty(StdDeserializer.java:604) at org.codehaus.jackson.map.deser.StdDeserializer.handleUnknownProperty(StdDeserializer.java:590) at org.codehaus.jackson.map.deser.BeanDeserializer.handleUnknownProperty(BeanDeserializer.java:689) at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:514) at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350) at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:120) at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:97) at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:258) at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:362) at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:494) at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350) at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:120) at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:97) at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:258) at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:362) at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:494) at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350) at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:258) at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:362) at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:494) at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350) at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402) at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602) at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:72) ... 20 more

smith1511 commented 11 years ago

I should add that I don't believe publicURL2 is valid in the service/endpoint schema so technically HP should fix their implementation.

However being realistic it might make sense to relax the client restrictions, ensuring that required fields are present, but ignoring any additional.