taniwallach
commented
5 years ago Below is a sample Perl script which creates a tests the basic idea of the code change. It
- Creates a sample OAuth signed "message" which includes some UTF8 Hebrew data.
- Checks that the message can be verified. (Passes)
- Break the message data by applying
utf8::encode()to the value which has UTF8 data in it. - Attempts to verify the tampered message data. (Fails and triggers the multi-byte character warning message from
Net::OAuth.) - Sets up some things so that the code added to
webwork2/lib/WeBWorK/Authen/LTIAdvanced.pmcan be run to "fix" the message. - Includes and runs the code to "fix" the message.
- Attempts to verify the "fixed" message data. (Passes.)
This seems to be a good indication that this is both the setting triggering the bug, and that the new code should fix it.
#!/usr/bin/perl -w
use strict;
use warnings;
use utf8;
use Net::OAuth;
my $HebrewString = "חסר קורס מהרשימה";
# ======================================================================
my $request_url = 'http://testing.123/foo';
my $consumer_key = 'my_sample_consumer_key';
my $consumer_secret = '123456789';
my %defaults = (
nonce => int( rand( 2**32 ) ),
timestamp => time,
consumer_key => $consumer_key,
consumer_secret => $consumer_secret,
# callback => $self->callback,
signature_method => 'HMAC-SHA1',
request_method => 'GET',
);
$defaults{protocol_version} = Net::OAuth::PROTOCOL_VERSION_1_0A ;
# ======================================================================
my $request = Net::OAuth->request('consumer')->new(
%defaults,
request_url => $request_url,
extra_params => {
samplePayload => "This has Hebrew in UTF8 in it $HebrewString",
foo => 'bar'
},
);
$request->sign;
my $post_body = $request->to_post_body ; # a application/x-www-form-urlencoded POST body
print "CREATED $post_body\n\n";
# ======================================================================
# Try to verify it
my %request_hash = ( %{$request->to_hash} );
my $test_request;
$test_request = Net::OAuth->request("request token")->from_hash( \%request_hash,
request_url => $request_url,
request_method => "GET",
consumer_secret => $consumer_secret,
);
print "About to decode the OAuth request\n";
my $decode_body = $test_request->to_post_body ; # a application/x-www-form-urlencoded POST body
print "VERIFY ORIGINAL $decode_body\n";
if ( $test_request->verify ) {
print "\tOAuth verification passed\n\n\n";
} else {
print "\tOAuth verification FAILED\n\n\n";
}
# ======================================================================
# Force the samplePayload payload field into UTF-8 BEFORE it gets sent to OAuth for handling.
# This intentionally breaks it.
print "\n\nWe now BREAK the samplePayload field by encoding it into UTF8 instead of Perl characters. This will break the verification and triugger the \"multi-byte character\" warning message from Net::OAuth.\n\n\n\n";
utf8::encode($request_hash{samplePayload});
$test_request = Net::OAuth->request("request token")->from_hash( \%request_hash,
request_url => $request_url,
request_method => "GET",
consumer_secret => $consumer_secret,
);
print "About to decode the OAuth request\n";
$decode_body = $test_request->to_post_body ; # a application/x-www-form-urlencoded POST body
print "VERIFY TAMPERED $decode_body\n";
if ( $test_request->verify ) {
print "\tOAuth verification passed\n\n\n";
} else {
print "\tOAuth verification FAILED\n\n\n";
}
# ======================================================================
# Try to fix any fields which seem to have UTF8 already
print "\n\nWe now attempt to fix the message using the code added to webwork2/lib/WeBWorK/Authen/LTIAdvanced.pm after setting up a suitable set of configuration settings for it to be used.\n\n\n\n";
# First put in some things, so we can test with the code added to
# webwork2/lib/WeBWorK/Authen/LTIAdvanced.pm
my %ce;
$ce{allow_WW_LTI_code_to_fix_what_appear_as_multi_byte_characters} = 1;
my $ce = \%ce;
sub debug { print( my $msg = shift ); }
# The code below is that added to webwork2/lib/WeBWorK/Authen/LTIAdvanced.pm
# except for some leading spaces deleted from the first and last line.
if ( defined($ce->{allow_WW_LTI_code_to_fix_what_appear_as_multi_byte_characters})
and ( $ce->{allow_WW_LTI_code_to_fix_what_appear_as_multi_byte_characters} == 1 ) ) {
my ( $key, $val );
foreach $key ( keys( %request_hash ) ) {
$val = $request_hash{$key} ;
if ( ($val =~ /[\x80-\xFF]/ and !utf8::is_utf8($val)) ) {
debug("Key ${key} : Detected what appears to be a UTF8 multi-byte characters which would have been passed to OAuth and triggered an error. Applying utf8::decode to the relevant value.\n");
utf8::decode($request_hash{$key});
}
}
}
$test_request = Net::OAuth->request("request token")->from_hash( \%request_hash,
request_url => $request_url,
request_method => "GET",
consumer_secret => $consumer_secret,
);
print "About to decode the OAuth request\n";
$decode_body = $test_request->to_post_body ; # a application/x-www-form-urlencoded POST body
print "VERIFY FIXED TAMPERED $decode_body\n";
if ( $test_request->verify ) {
print "\tOAuth verification passed\n\n\n";
} else {
print "\tOAuth verification FAILED\n\n\n";
}
mgage
nmoller
dlglin
There are several reports in the forums about LTI authentication failing when the LTI source (the external LMS) is sending multi-byte UTF8 characters as part of the request parameters.
This is an issue with releases 2.13 and 2.14 and possibly with earlier releases which already had LTI support. It is not critically connected to the internationalization/localization project, as even regular "English only" WW sites may have student names which contain accented (or other special) characters which are encoded by multiple bytes in UTF8 which is used as the default encoding for the transfer of the form data, including the LTI authentication data.
The core issue
The issue is related to the fact the the Perl Net::OAuth module used to verify the LTI authentication process requires the input to be in "internal Perl character encoding" and not encoded as bytes in UTF8. See: https://metacpan.org/pod/release/KGRENNAN/Net-OAuth-0.28/lib/Net/OAuth.pm and in particular the section called
I18Nnear the bottom which states:Beta version of a fix is ready to be tested by people with a working WW-LMS LTI setup.
After looking into the issue, I have made a beta-version of a fix available.
utf8::decode()to values in the%request_hashwhich match the same regexp as Net::OAuth uses to trigger the error message.$allow_WW_LTI_code_to_fix_what_appear_as_multi_byte_charactersand an explanation and a comment out line to do this were added toconf/authen_LTI.conf.distin the feature branch.Testing
Testing requires a course which allows LTI authentication, and connecting from an LMS properly configured to access the WeBWorK server via LTI.
lib/WeBWorK/Authen/LTIAdvanced.pmlib/WeBWorK/Authen/LTIBasic.pm$allow_WW_LTI_code_to_fix_what_appear_as_multi_byte_characters=1inconf/authen_LTI.conf(or in a course specific config file).