BB port: ConnectionRequest fails with err code 403 in network w/o BB service

[GoogleCodeExporter]

What steps will reproduce the problem?
1. build an BB app using ConnectionRequest class, with CN1 signing certificate 
(by default)
2. try to run it on network of a carrier without BB service support (I have BB 
Curve 8310 in a Russian GSM network with BB service disabled). 3. Start the app 
and see that HTTP request fails with err code 403 (forbidden), while other apps 
can work OK (BB Maps, Google maps, Opera mini, Waze, my own networking app)

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
BB Curve 8310, Russian GSM network Megafon

Please provide any additional information below.

I think that the matter is following - see this code in 
BlackBerryImplementation.java:

---
    public Object connect(String url, boolean read, boolean write) throws IOException {
        if (currentAccessPoint != null) {
            url += ";ConnectionUID=" + currentAccessPoint;
            if (deviceSide) {
                url += ";deviceside=true";
            }
        }
        return connectImpl(url, read, write);
    }

---
In my working networking app I use open source HttpLibrary: 
http://supportforums.blackberry.com/t5/Java-Development/Sample-HTTP-Connection-c
ode-and-BIS-B-Access/td-p/653175
For my network conditions it adds these transport keys:
url += ";deviceside=true";
But in CN1 BB implementation we see that it will be added only when APN is 
explicitly specified via NetworkManager. But it should work without that, as 
HttpLibrary does.
I tried to specify APN (it's "internet" for my carrier) via 
NetworkManager.setCurrentAccessPoint(), but it didn't help. I didn't debug CN1 
BB port - it could happen if your code doesn't calculate deviceSide flag 
properly (or may ";ConnectionUID=" should be used, as HttpLib would add ";apn=" 
for this case, I think). I don't know, and my BB knowledge is very limited.
The connection was made via GPRS (BB 8310 doesn't have WiFi), OS 4.5 on board. 
I used CN1 build for 4.2.
I have proper APN set in BB phone settings. My networking app (with HttpLib) 
worked OK even without ServiceBooks (I installed them separately few days ago 
to get access to BB features such as browser, but the app worked before that as 
well).

I think that CN1 generates connection transport keys differently then HttpLib 
does, for this case (no BB services in carrier's network). Please fix.

Original issue reported on code.google.com by gmi...@gmail.com on 13 Apr 2012 at 9:32

[GoogleCodeExporter]

Correction:
... (or may be ";ConnectionUID=" should _not_ be used, as HttpLib would add 
";apn=" for this case, I think) ...

Original comment by gmi...@gmail.com on 13 Apr 2012 at 9:34

[GoogleCodeExporter]

Did you see the code within NetworkManager (line 312) the inner class 
AutoDetectAPN?

That class is supposed to try connecting to a URL on our server to see if it 
has access and pick the right APN. I think ConnectionUID is fine if it has the 
right value. Its possible we need to also flip the deviceSide flag to on during 
our testing.

Original comment by shai.almog on 14 Apr 2012 at 4:30

[GoogleCodeExporter]

About AutoDetectAPN - it's an interesting approach, but it may have drawbacks - 
 extra network connections to http://codename-one.appspot.com/t.html. Also 
performing test connections takes time.
I can't say for generic rule, but for my particular case it is better to try 
first ";deviceside=true" without APN setting.

BB Curve 8310 is an EDGE device, no 3G, so seems this code sets 'deviceSide' to 
false:

    public void setCurrentAccessPoint(String id) {
        currentAccessPoint = id;
        int t = getAPType(id);
        deviceSide = t == NetworkManager.ACCESS_POINT_TYPE_NETWORK3G || t == NetworkManager.ACCESS_POINT_TYPE_WLAN;
    }

Why isn't it set for ACCESS_POINT_TYPE_NETWORK2G?

Original comment by gmi...@gmail.com on 14 Apr 2012 at 8:35

[GoogleCodeExporter]

Is that still realistic?
Are there still devices like this shipping? 

Generally we don't have a choice with RIM, a device might be misconfigured and 
people still expect it to work. I couldn't find any other way to get networking 
to work with a RIM device. 

Original comment by shai.almog on 14 Apr 2012 at 8:47

[GoogleCodeExporter]

Of course EDGE devices are not shipped anymore, it's an old device. But all 
other apps seem to work OK on it, including those built on HttpLibrary.

Generally I do not mind testing APNs via  
http://codename-one.appspot.com/t.html, and probably it may be useful in some 
cases, you have more experience here.
What about setting deviceSide=true for ACCESS_POINT_TYPE_NETWORK2G case? Do you 
mean that it may conflict with something? I do not think that 'diveceside' flag 
in url  should depend on 2G/3G network type.
I haven't seen phones with 2G/3G switch in APN settings, so I wonder how a 2G 
APN setting could affect this flag on modern 3G phones.

So, my suggestion is to make this change:
deviceSide = t == NetworkManager.ACCESS_POINT_TYPE_NETWORK2G || t == 
NetworkManager.ACCESS_POINT_TYPE_NETWORK3G || t == 
NetworkManager.ACCESS_POINT_TYPE_WLAN;
Any objections?

Original comment by gmi...@gmail.com on 14 Apr 2012 at 9:11

[GoogleCodeExporter]

I don't think that change will solve the problem (have you tried it). I think 
the problem is we don't even search for a 2G network. 

Original comment by shai.almog on 14 Apr 2012 at 10:14

[GoogleCodeExporter]

There are lots of areas in Canada with only 2G service still, including most of 
northern Ontario, I see my Android drop to it on occasion here in Southern 
Ontario where we even have 4G.

My BB is also the 8320, and a number of people I know have similar models, I 
think it's worth supporting still, just as much as J2ME.  They sell lots of 
them around here at the 'factory outlet' stores.  

Original comment by 1815...@coolman.ca on 14 Apr 2012 at 10:02

[GoogleCodeExporter]

Yes, it's interesing - what happens if connection is made on 3G device, when 
network degrades to 2G - will current code work OK? I also observe such cases 
quite often, especially in areas of bad coverage.

Original comment by gmi...@gmail.com on 14 Apr 2012 at 10:12

[GoogleCodeExporter]

I added 2G support for this feature. 
It won't allow changing a network in runtime. 

Original comment by shai.almog on 15 Apr 2012 at 4:36

• Changed state: Fixed

[GoogleCodeExporter]

I've just tested - works the same (HTTP error 403). Were the changes applied to 
the build server?
I do not build BB ports on my PC - I haven't prepared buildchain for it yet...

Original comment by gmi...@gmail.com on 15 Apr 2012 at 9:30

[GoogleCodeExporter]

They were only applied to the server now

Original comment by shai.almog on 15 Apr 2012 at 10:27

[GoogleCodeExporter]

Tested again once more - works the same (error 403) ((
Seems that I'll need to build BB port on my PC, to debug what the matter is...

Original comment by gmi...@gmail.com on 15 Apr 2012 at 10:54

[GoogleCodeExporter]

Finally I was able to debug the app on my BB 8310, and here goes some 
sadditional info about the problem. I added logging to 
BlackberryImplementation.java to see what BB transport keys are added to URL:
---
    public Object connect(String url, boolean read, boolean write) throws IOException {
        if (currentAccessPoint != null) {
            url += ";ConnectionUID=" + currentAccessPoint;
            if (deviceSide) {
                url += ";deviceside=true";
            }
        }
        Log.p("connect to "+url);
        return connectImpl(url, read, write);
    }
---
Here goes the output:
---
[Network Thread] 0:0:0,4 - connect to http://codename-one.appspot.com/t.html
[Network Thread] 0:0:5,584 - connect to 
http://codename-one.appspot.com/t.html;ConnectionUID=S TCP-WiFi;deviceside=true
[Network Thread] 0:0:15,284 - connect to 
http://codename-one.appspot.com/t.html;ConnectionUID=WAP2 trans;deviceside=true
[Network Thread] 0:0:17,56 - connect to 
http://codename-one.appspot.com/t.html;ConnectionUID=MMS trans;deviceside=true
[Network Thread] 0:0:21,32 - connect to 
http://www.upsilon-studio.com/eskimi/TOS.txt;ConnectionUID=MMS 
trans;deviceside=true
[Network Thread] 0:0:23,284 - exception java.lang.Exception : Bad response code 
403: Forbidden
---

Neither of the requests succeeded, though just adding ";clientside=true" would 
do. 

And if it have succeed with WAP2 transport - I would be probably billed by 
carrier for wap traffic (which is very expensive) though I have unlim plan for 
regular TCP/IP traffic via APN?

What we see - 
1) original request (from app side) was made to 
http://www.upsilon-studio.com/eskimi/TOS.txt
2) CN1 tries to detect transport via http://codename-one.appspot.com/t.html 
(could it be better to consider deviceside flag here?)
3) CN1 tries to connect via several transports (I see them in ServiceBook on 
device)
4) finally it tries the original URL (requested by the app) via the last 
protocol though it already failed before (seems unnecessary)

If I specify APN name
NetworkManager.getInstance().setCurrentAccessPoint("internet");
I get this in logs:
[Network Thread] 0:0:0,0 - connect to 
http://codename-one.appspot.com/t.html;ConnectionUID=internet
[Network Thread] 0:0:0,760 - connect to 
http://www.upsilon-studio.com/eskimi/TOS.txt;ConnectionUID=internet
[Network Thread] 0:0:1,184 - exception java.io.IOException : Tunnel failed

We see that CN1 doesn't add deviceside flag in this case, and doesn't use 
";apn=" flag (uses ";ConnectionUID=" instead), which is incorrect.

With HttpLibrary I use this code:
---
    public static String makeBlackberryURL(String url) {
        URLFactory uu = new URLFactory(url);
        TransportDetective td = new TransportDetective();
        int tr = td.getBestTransportForActiveCoverage();
        switch(tr) {
        case TransportDetective.TRANSPORT_TCP_WIFI:
            url = uu.getHttpTcpWiFiUrl();
            break;
        case TransportDetective.TRANSPORT_MDS:
            url = uu.getHttpMdsUrl(false);
            break;
        case TransportDetective.TRANSPORT_BIS_B:
            url = uu.getHttpBisUrl();
            break;
        case TransportDetective.TCP_CELLULAR_APN_SERVICE_BOOK:
        case TransportDetective.TRANSPORT_TCP_CELLULAR:
            url = uu.getHttpTcpCellularUrl(null, null, null);
            break;
        }
        return url;
    }
---

I'm attaching HttpLibrary files just in case.

Shai - I think that this ticket needs to be reopened.

Original comment by gmi...@gmail.com on 17 Apr 2012 at 1:50

Attachments:

• TransportDetective.java
• URLFactory.java

[GoogleCodeExporter]

Do you have a more concrete suggestion as to how this should be fixed?
I took the ConnectionUID code from the address book but we need something 
generic to solve this.

Original comment by shai.almog on 17 Apr 2012 at 1:56

• Changed state: Accepted

[GoogleCodeExporter]

correction: "... just adding ";deviceside=true" would do"

Original comment by gmi...@gmail.com on 17 Apr 2012 at 1:57

[GoogleCodeExporter]

Currently I'm going to fix CN1 sources in this way:

1) disable BlackberryImplementation.shouldAutoDetectAccessPoint()
2) add the code I mentioned (with HttpLibary) to 
BlackberryImplementation.connect()

Original comment by gmi...@gmail.com on 17 Apr 2012 at 2:02

[GoogleCodeExporter]

I'll probably detect transport on each connect - may be useful in cases when 
network conditions change (for example when user connected to WiFi spot while 
app is working)

Original comment by gmi...@gmail.com on 17 Apr 2012 at 2:04

[GoogleCodeExporter]

There are two problems with this: According to the RIM documentation the best 
transport might not work.
Second issue is that there is no copyright or ownership on the code you 
submitted. Where is it from? What's the license?

Original comment by shai.almog on 17 Apr 2012 at 6:13

[GoogleCodeExporter]

I already provided link to HttpLibrary: 
http://supportforums.blackberry.com/t5/Java-Development/Sample-HTTP-Connection-c
ode-and-BIS-B-Access/td-p/653175 
AFAIK it's in public domain.
Yes, I agree that it has problem, one that you mentioned, and also it doesn't 
consider WAP and WAP2 transports, which operate with ConnectionUI (as I can 
judge from source code).
Testing connections like you do, makes sense. But there is a wider set of 
possible transports. And it would be good to start testing from the best 
transport first.
Looks like there may be several transports available at the same time, and 
probably there should be an API to select one by user.

Original comment by gmi...@gmail.com on 17 Apr 2012 at 6:28

[GoogleCodeExporter]

"it doesn't consider WAP and WAP2 transports" - I meant that my code (with the 
switch statement) doesn't consider them. But they are supported by HttpLibrary.

Original comment by gmi...@gmail.com on 17 Apr 2012 at 6:31

[GoogleCodeExporter]

We can add these options in the implementation, that should probably be easy 
enough.

Original comment by shai.almog on 17 Apr 2012 at 6:35

[GoogleCodeExporter]

Yes, I hope so. Actually BB networking is a almost a dark area for me.
By the way - HttpLibrary doesn't compile againt SDK 4.2, but I compiled it 
against 4.5.
It fixed the issue on my device (I think is transport-related issue, so may 
affect newer BB devices as well), but generally needs a better solution 
considering the items you mentioned.

Original comment by gmi...@gmail.com on 17 Apr 2012 at 7:34

[GoogleCodeExporter]

I committed this code please check that it solves the issue.
Chen please check it on our blackberry.

FYI we already have an API to pick a connection explicitly the Access Point API 
in network manager.

Original comment by shai.almog on 18 Apr 2012 at 4:29

• Changed state: Fixed

[GoogleCodeExporter]

I have tested - it works OK for me too. Thanks

Original comment by gmi...@gmail.com on 19 Apr 2012 at 12:03