Not sure how you test, but it seems to work as expected for me:
Album is public, anonymous user can see it
Change album to private using admin account; anonymous user can't see it
Create a guest token; anonymous user can't see it
Anonymous user uses the Guest Token link; he can see the album
Maybe this is what made you think the album was now public:
If you try to load the album from a browser that used the guest token in the past, that album will be visible.
That is because using the guest token sets a cookie that gives that browser access to the private album.
Open a new incognito window (or use a different browser); you should not be able to open the private album until you use the guest token link.
Guest Token seems to make the folder public at the same time.