Transfer encoding request or response issue

[gopisraj14]

Hi,

Transfer request and response will not maintain the original format of text.

Please let me know the procedure to retain the original format of text(file may be .txt, .bmp, .jpg, etc.,)

Regards,

Gopi S

[ben-crowhurst]

Please present your service and client code for examination. My initial guess is that you fail to remove the length\r\n from the response stream. It's hard to tell with out seeing your code base.

See Transfer-Encoding for the details of the data transfer mechanism.

[gopisraj14]

Hi,

Thank you for your email.

Please find the attached sample code.

My requirement: I need upload or download any type of file using RESTBED service (file types: .txt, .jpg, .png, .zip etc.,)

Regards,

Gopi S

On Wed, Feb 24, 2016 at 12:33 AM, Corvusoft notifications@github.com wrote:

Please present your service and client code for examination. My initial guess is that you fail to remove the length\r\n from the response stream. It's hard to tell with out seeing your code base.

— Reply to this email directly or view it on GitHub https://github.com/Corvusoft/restbed/issues/74#issuecomment-187843180.

[ben-crowhurst]

@gopisraj14, I can't see the attachment? Please submit your code via the issue tracker.

[gopisraj14]

Hi Ben,

I am unable to upload the code in github.

shared the code to info@corvusoft.co.uk email.

Regards,

Gopi S

On Thu, Feb 25, 2016 at 12:29 AM, Ben Crowhurst notifications@github.com wrote:

@gopisraj14 https://github.com/gopisraj14, I can't see the attachment? Please submit your code via the issue tracker https://github.com/Corvusoft/restbed/issues/74#

— Reply to this email directly or view it on GitHub https://github.com/Corvusoft/restbed/issues/74#issuecomment-188406045.

[callum-kirby]

@gopisraj14 Due to our security policies, I'm afraid I can't download and open the supplied attachment via info@coruvsoft.co.uk. What issues are you having specifically with presenting your code on this form?

I can help you with any issues you maybe having via GitHub. Alternatively please see the Transfer-Encoding examples request and response.

Documentation RFC Section 3.6 Wikipedia Transfer Encoding

[gopisraj14]

Hi,

The sample request and response code itself not maintaining format, Please check it.

Regards,

Gopi S

On Fri, Feb 26, 2016 at 12:13 AM, Callum Kirby notifications@github.com wrote:

@gopisraj14 https://github.com/gopisraj14 I'm afraid I can't download and open the supplied attachment via info@coruvsoft.co.uk, due to our security policies. What issues are you having specifically with presenting your code on this form?

I can help you with any issues you maybe having via GitHub. Alternatively please see the Transfer-Encoding examples request https://github.com/Corvusoft/restbed/blob/master/example/transfer_encoding_request/source/example.cpp and response https://github.com/Corvusoft/restbed/blob/master/example/transfer_encoding_response/source/example.cpp .

Documentation RFC Section 3.6 https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6 Wikipedia Transfer Encoding https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol

— Reply to this email directly or view it on GitHub https://github.com/Corvusoft/restbed/issues/74#issuecomment-188923673.

[callum-kirby]

In what way is it not maintaining the format? Please provide more detail in your response other than "It's not working".

[ben-crowhurst]

@gopisraj14 Do you have an email address I can contact you privately? @tataelxsi.co.in perhaps?

[gopisraj14]

The resoucee file called request.txt is formatted file when we stream the content to server -- at the server end its not displaying in original format.

On Fri, Feb 26, 2016 at 1:21 AM, Callum Kirby notifications@github.com wrote:

In what way is it not maintaining the format? Please provide more details than "It's not working".

— Reply to this email directly or view it on GitHub https://github.com/Corvusoft/restbed/issues/74#issuecomment-188950576.

[callum-kirby]

The error must be in code you have written. Both examples in the source tree work as expected.

Until you post the offending code on Github, there is really little else I can do to assist you.

[gopisraj14]

Hi,

I have attached code. download.txt and upload.txt are cpp files, please rename it to .cpp

Request

#include <string>
#include <memory>
#include <cstring>
#include <cstdlib>
#include <ciso646>
#include <iostream>
#include <restbed>
#include <fstream>
#include <iostream>
#include <vector>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>

using namespace std;
using namespace restbed;

void post_method_handler( const shared_ptr< Session > );
void read_chunk( const shared_ptr< Session >, const Bytes& );
void read_chunk_size( const shared_ptr< Session >, const Bytes& );

int main( const int, const char** )
{
    auto resource = make_shared< Resource >( );
    resource->set_path( "/upload" );
    resource->set_method_handler( "POST", post_method_handler );

    auto settings = make_shared< Settings >( );
    settings->set_port( 1984 );
    settings->set_default_header( "Connection", "close" );

    Service service;
    service.publish( resource );
    service.start( settings );

    return EXIT_SUCCESS;
}

void post_method_handler( const shared_ptr< Session > session )
{
    const auto request = session->get_request( );

    if ( request->get_header( "Transfer-Encoding", String::lowercase ) == "chunked" )
    {
        session->fetch( "\r\n", read_chunk_size );
    }
    else if ( request->has_header( "Content-Length" ) )
    {
        int length = 0;
        request->get_header( "Content-Length", length );

        session->fetch( length, [ ]( const shared_ptr< Session > session, const Bytes& )
        {
            const auto request = session->get_request( );
            const auto body = request->get_body( );

            fprintf( stdout, "Complete body content: %.*s\n", static_cast< int >( body.size( ) ), body.data( ) );
            session->close( OK );
        } );
    }
    else
    {
        session->close( BAD_REQUEST );
    }
}

void read_chunk_size( const shared_ptr< Session > session, const Bytes& data )
{
    if ( not data.empty( ) )
    {
        const string length( data.begin( ), data.end( ) );

        if ( length not_eq "0\r\n" )
        {
            const auto chunk_size = stoul( length, nullptr, 16 ) + strlen( "\r\n" );
            session->fetch( chunk_size, read_chunk );
            return;
        }
    }

    session->close( OK );

    const auto request = session->get_request( );
    const auto body = request->get_body( );

    fprintf( stdout, "Complete body content: %.*s\n", static_cast< int >( body.size( ) ), body.data( ) );

    ofstream filePtr;
        filePtr.open("gopi.txt", std::ofstream::out | std::ofstream::trunc | std::ofstream::binary);
        filePtr << body.data();
        filePtr.close();
}

void read_chunk( const shared_ptr< Session > session, const Bytes& data )
{
    cout << "Partial body chunk: " << data.size( ) << " bytes" << endl;

    session->fetch( "\r\n", read_chunk_size );
}

Response

#include <string>
#include <chrono>
#include <memory>
#include <cstdlib>
#include <restbed>
#include <unistd.h>

#include <fstream>
#include <iostream>
#include <vector>

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>

using namespace std;
using namespace restbed;

void post_method_handler( const shared_ptr< Session > session )
{
ulong len = 0;
        session->yield( OK, "", { { "Connection", "keep-alive" } } );

        //string file_path = "install.log";
        string file_path = "file.txt";
        size_t chunk_size = 1024 * 32;
        ifstream file(file_path.c_str(), ifstream::binary);

        /* basic sanity check */
        if (not file)
        {
                cerr << "file: " << file_path.c_str() << " failed to open" << endl;
                return;
        }

        /* *NIX way to get file size without seeking to the end and back */
        struct stat filestatus;
        stat(file_path.c_str(), &filestatus);

        size_t total_size = filestatus.st_size;

        /* atoi may fail and leave us with an undefined chunk size*/
        if (not (chunk_size > 0))
        { chunk_size = 1 * 1024 * 1024; }
                cout << "using chunk size: " << chunk_size << endl;

        /* on to the actual algorithm */
        size_t total_chunks = total_size / chunk_size;
        size_t last_chunk_size = total_size % chunk_size;

        if (last_chunk_size != 0) /* if the above division was uneven */
        {
        ++total_chunks; /* add an unfilled final chunk */
}
        else /* if division was even, last chunk is full */
        {
                last_chunk_size = chunk_size;
        }
        /* the loop of chunking */
        for (size_t chunk = 0; chunk < total_chunks; ++chunk)
        {
                size_t this_chunk_size =
                chunk == total_chunks - 1 /* if last chunk */
                ? last_chunk_size /* then fill chunk with remaining bytes */
                : chunk_size; /* else fill entire chunk */

                /* if needed, we also have the position of this chunk in the file
                size_t start_of_chunk = chunk * chunk_size; */

                /* adapt this portion as necessary, this is the fast C++ way */
                vector<char> chunk_data(this_chunk_size);
                file.read(&chunk_data[0], /* address of buffer start */
                this_chunk_size); /* this many bytes is to be read */

                /* do something with chunk_data before next iteration */
                cout << "chunk #" << chunk << endl;
                /*for (const auto c : chunk_data) // I like my C++11 extensions
                {
                        cout << c;
                }*/
                std::string data_str = std::string(chunk_data.begin(), chunk_data.end());
                char len_str[100] = {0};
                sprintf(len_str,"%ld", data_str.length());
                len +=data_str.length();
                session->yield( OK, data_str, { {"Content-length", len_str }, { "Connection", "keep-alive" } } );
                sleep(5);

                //cout << endl;
        }
        cout<<"Total String length "<<len<<endl;
        session->close( OK, "OK", { { "Connection", "close" } } );
        return;

}

int main( const int, const char** )
{
    auto resource = make_shared< Resource >( );
    resource->set_path( "/download" );
    resource->set_method_handler( "POST", post_method_handler );

    auto settings = make_shared< Settings >( );
    settings->set_port( 1984 );
    settings->set_default_header( "Connection", "close" );

    Service service;
    service.publish( resource );
    service.start( settings );

    return EXIT_SUCCESS;
}

Please help me on it

request.txt

[ben-crowhurst]

OK, I think I see the issue. So lets just clarify a few points:

1. What are you building?
2. Is this homework or for work?
3. Are you attempting to request a file and then stream it's contents back to the client?

[gopisraj14]

Hi,

I am working on one product development (still doing some research).

requirement:

I need to upload a file to server and download from the server (any format files like .txt, .zip, .png, .jpg etc.,)

Regards,

Gopi S

On Fri, Feb 26, 2016 at 2:48 PM, Ben Crowhurst notifications@github.com wrote:

OK I think I can see the issue. So let me just clarify a few points.

1. What are you building?
2. Is this homework or for work?
3. Are you attempting to request a file an then stream its contents back to the client?

— Reply to this email directly or view it on GitHub https://github.com/Corvusoft/restbed/issues/74#issuecomment-189180904.

[ben-crowhurst]

@gopisraj14 Thanks for that. Could you possibly confirm the name of your employer before I highlight the major issues?

[gopisraj14]

HI, I am working for ComX Technologies India private limited. www.comxtech.com

Regards,

Gopi S

[ben-crowhurst]

@gopisraj14 Thanks for that information.

The first major issue is that the HTTP Request/Response cycle only expects a single status code returned (certain areas do violate this, for example HTTP header Expect: 100-Continue). Yet in your Response post_method_handler you yield OK repeatedly and then call close with an OK status again.

Please see RFC2616 for HTTP specification details.

Due to the condition of the supplied code it would be far easier to rewrite it for you from the ground up than explain all of the errors. This will take several hours and the office won't be open again until Monday. I'll look to have someone respond with a working example next week at some point.

Enjoy your weekend.

[ben-crowhurst]

@gopisraj14 Trying to send you Transfer-Encoding code examples via gopisraj14@comxtech.com and gopisraj@comxtech.com but I keep getting rejected?

[ben-crowhurst]

Mr Gopi Sundar Raj please provide me with a private email so we may discuss what is going on here.

[gopisraj14]

please send it to gopi@comxtech.com

On Fri, Feb 26, 2016 at 4:00 PM, Ben Crowhurst notifications@github.com wrote:

Gopi Sundar Raj please provide me with a private email so we may discuss what is going on here.

— Reply to this email directly or view it on GitHub https://github.com/Corvusoft/restbed/issues/74#issuecomment-189209320.