Australian Transport Card - Opal (Sydney, NSW)

[phiali]

I'd really love to see support for the Opal card (www.opal.com.au). Scanning a card shows that there is an open area and I've managed to find out that this contains the balance.

With that data being unencrypted do we stand a better chance of trying to integrate this? I'm more than happy to dump some data along with what I know my balance of the card is.

[sburford]

Sydney Opal cards are DesFire EV1 cards that have an app 0x534531. File 0x07 is the only generally readable file within this application, all other files are protected by authentication. File 0x07 is 16 bytes long. For example card serial 3085 2200 4381 2318: 2F DA 42 00 28 02 80 01 01 30 C8 6C 88 63 D1 57

To interpret this file I start by converting it to a bit stream: For each byte: For bit in 0x01 0x02 ... 0x80: If byte and bit == bit: output a 1 else: output a 0

The first 32 bits are part of the serial number printed on the reverse of the card, least significant bit first: 0x0042DA2F = 4381231 decimal The next 5 bits are the check digit that appears at the end of the serial: 0x08 So this cards serial is the constant 308522 followed by the %09d 004381231 followed by the check digit 8. This matches the serial given above (3085 2200 4381 2318)

The next 11 bits are a count of how many times this card has been used (including top ups): 17 decimal The next 5 bits are zero. The next 15 bits are the balance in cents. $20.60 here. The next 1 bit indicates that the previous transaction reversed the charge. For example, the user tapped on then immediately tapped off to indicate that they were not actually going to travel. The next 5 bits are flags. If they are all set the balance is actually negative. A negative balance is calculated as (-1 * ((balance + 1) ^ 0x7FFF)) The next 15 bits are the date of the previous transaction in days since 1/1/1980. The next 11 bits are the time of the previous transaction in minutes since midnight. The next 4 bits are mode of transport: 00: Kiosk, for example topping up the balance at a shop. 08: Train 09: Ferry 0A: Bus The next bit is 0 if a trip is in progress, 1 otherwise The next 2 bits are the result of the previous tap: 00: In transit 01: Trip complete 02: Reversed charge 03: Already tapped on If mode is zero and trip in progress is zero and tap result is zero the card was most recently used to top up the balance. The next bit is unknown. The next 4 bits represent how many journeys have been taken this week. The next 16 bits are some kind of checksum

[micolous]

I've started writing a patch for this, I need to clean it up and write some documentation of the format.

I found the easiest way to parse the data blob is to reverse the bytestream, this way you don't have to worry about the byte order on the multi-byte values. I've tested this with a couple of cards and it seems to work for me. Farebot's utility functions have made short work of it. :smile:

I haven't attempted to work on the checksum.

My code is based on PR #98 as I'm working with Android Studio 1.x and the code would not build otherwise.

I'm still trying to get my head around some of the API in Farebot and making this display meaningful data.

[auscompgeek]

In other news, the official Opal app can now read this data: https://play.google.com/store/apps/details?id=au.com.opal.travel

Opal Travel 1.2: This update brings the ability to scan your Opal card directly in the app using the NFC capabilities on your phone, allowing you to view your balance and weekly travel reward.

[micolous]

My branch now decodes all fields from the card, and I've written a wiki page with notes about the card format.

I'm interested in what the other usage types are, because there's many missing from the list. I've included the ones I have observed in the page.

Light Rail and Ferry transit type codes are the same. But this could also be that Light Rail has no code, and copies what was on the card previously.

[sburford]

Great work on the modes and usage types!

On Sat, Jul 4, 2015 at 2:09 AM, Michael Farrell notifications@github.com wrote:

My branch now decodes all fields from the card, and I've written a wiki page https://github.com/codebutler/farebot/wiki/Opal with notes about the card format.

I'm interested in what the other usage types are, because there's many missing from the list. I've included the ones I have observed in the page.

Light Rail and Ferry transit type codes are the same. But this could also be that Light Rail has no code, and copies what was on the card previously.

— Reply to this email directly or view it on GitHub https://github.com/codebutler/farebot/issues/105#issuecomment-118384023.

Sean Burford sburford@google.com lg​​t᠎m, a᠎p​pro​v᠎al.

[auscompgeek]

@micolous @sburford I reverse engineered the Opal Travel app to get all those usage types. I never realised how horribly ferries were handled by Opal until I saw it…

[micolous]

@auscompgeek I think you'll find the issue is how amazingly convoluted TfNSW's fare structure is. :scream:

I notice that you've removed any mention of Light Rail explicitly mentioned in your details there. Based on my tests it seems to use the same code as the Ferry.

I don't normally travel using Opal so haven't tested tapping off the trains at Central followed by tapping on the Light Rail to see if the code is unchanged by LR.

This does lead to an interesting topic of what happens if you tap on at The Star LR then tap off at Casino Wharf, given that they're the same travel mode (but one has a much higher fare).

Given how the ticketing works for the Manly Ferry (there is no tap-off) it makes total sense that it is implemented as it's own set of codes. This seems to be absent since your refactor.

[micolous]

Regarding codes 9 and 10 I've translated that to something which would make sense.

The only condition I think you'd see code 9 (no tap off) on is if you have a default fare charged from not tapping off, then recharge your card using a physical terminal.

Code 10 is easy to get, just try to exit any gated train station.

I don't think the article needs to repeat the non-reversed form -- all the bit-wise data reading is big-endian so reading it as anything but that (and doing byte swaps on each boundary) is a mess. When I was prototyping this out in Python it had the same issue.

I'll leave you to deal with the Manly Ferry, that's handled differently to everything else. You should be able to enter the gates and exit the gates at either side once you have a card scan. I picked one up from someone who catches the service regularly and it didn't have any of the journey completion codes I'd seen in other scans.

I'd also like to know the difference between code 5 and 6.

Non-city Ferry wharves have no gates so you can tap on and off to your heart's content. :smile:

Once you're happy with the translation of the action codes I'll pop it into my branch.

[auscompgeek]

I notice that you've removed any mention of Light Rail explicitly mentioned in your details there. Based on my tests it seems to use the same code as the Ferry.

That's definitely weird. Opal's activity statements display the train icon for light rail trips. They still charge the correct fare though.

The Opal reader code in the Opal Travel app does not mention light rail explicitly anywhere, leading me to believe that this is also how it's stored on the card.

Code 10 is easy to get, just try to exit any gated train station.

Ah, that'd make sense. I've never had a chance to encounter a default fare from doing that. Good call.

I'd also like to know the difference between code 5 and 6.

Same difference between codes 2 and 3.

Given how the ticketing works for the Manly Ferry (there is no tap-off) it makes total sense that it is implemented as it's own set of codes.

This is deliberate, I am simply going off the Opal Travel app here.

I don't think the article needs to repeat the non-reversed form -- all the bit-wise data reading is big-endian so reading it as anything but that (and doing byte swaps on each boundary) is a mess. When I was prototyping this out in Python it had the same issue.

Yeah, it seems like a mess trying to read the data in a forward fashion after reading the serial number. I originally rewrote the entire page to list it forwards, but then encountered the same issue myself.

I'll leave you to deal with the Manly Ferry, that's handled differently to everything else.

I have a feeling it's just a special case (in the Weekly Travel Reward handling) that's hardcoded into the gates. Opal deducts the default fare from your balance when you tap on. Since there's only one possible stop on the Manly Ferry service, it's already deducted the correct fare, so the readers display this when you tap on. You almost certainly won't get a tap off code unless you do a tap on reversal (or if you actually tap off).

[auscompgeek]

Wow, ok. The Opal Travel app actually displays "Ferry / Light Rail" for the ferry mode type. Screenshot courtesy of someone in the UNSW OpalSoc Facebook group:

[micolous]

Ah good, glad we're on the same page. I don't trust the application to have necessarily obvious or correct constants, given that the view layer can do whatever it likes.

I tapped on at Casino Wharf which marked my card with "new journey" (0x01), then tapping again set it to "tap on reversal" (0x0b). This is a different set of constants to what I saw with a friend who catches the Manly-CQ ferry, who got 0x04 once "tapped on".

There's no field in the freely-readable part of the card which indicates which station you're at, so it's not possible to otherwise flag the Manly-CQ ferry differently.

All the other ferries I've caught at CQ I've needed to tap off. Based on the constants you're providing and those code differences, I'd say that 0x04-0x06 are all Manly-CQ ferry specific codes.

[auscompgeek]

Manly-CQ

Oh, right. That's what the M stands for in the constants. >_<

[micolous]

I now have a card with a negative balance thanks to an anonymous donor. It looks like I made a mistake with negative values handling. :disappointed:

A card with a value of -$1.82 shows as +$20,969.70

[micolous]

Negative balances now work correctly. Implementation was fine, just had an off-by-1 issue.

[micolous]

I pushed a beta to the Play Store. You need to join my beta group, then after a few minutes you will have access to the beta via this link.

[micolous]

The upstream developer didn't seem interested merging my pull request (#110). I've disabled the beta restrictions on my fork. I'm not interested in wasting time on trying to get my changes merged when this project seems abandoned.

If you want Opal support, please download my fork from the Play Store: https://play.google.com/store/apps/details?id=au.id.micolous.farebot

[codebutler]

Hi everyone, this code has been merged and will be available in the next release. Thanks @micolous for all the work on this!

If you'd like earlier access please join the beta: https://play.google.com/apps/testing/com.codebutler.farebot

[ghost]

Heyo! I know this is an old thread, but I've just started on FareBot and looked to try my "school" Opal card with a lack of support. I believe there might be some data on there as I've checked on my Proxmark.