Asynchronous web service calls for File Connector

[GoogleCodeExporter]

Feature I would like to see:

It will be a very good value addition if the web services provided by File 
Connector are asynchronous.

Each web service should take the required operation and notification back 
details (like end point and authentication details)

Impact if not implemented:

Most of the file connector web services deal with disk IO operations and all of 
them are synchronous.

In a typical Cordys development/test/production environment all the web 
services are set with a time out value of 30 seconds.

There is every possibility for the File Connector web services to time out in 
case:

1. Dealing with large files.
2. Slow disk IO

Alternatives:

Original issue reported on code.google.com by udayku...@gmail.com on 2 Jul 2011 at 11:19

[GoogleCodeExporter]

This is a good enhancement. 
However it should be optional since asynchronity will add additional complexity 
when dealing with small files on a fast disk system.

Original comment by cor...@endt.info on 4 Jul 2011 at 7:53

[GoogleCodeExporter]

I agree with you. However, when the application running on production 
environment, I am not sure about what is fast and slow.

Original comment by udayku...@gmail.com on 4 Jul 2011 at 9:14

[GoogleCodeExporter]

This allways hard to decide you are rigt. It can only be determined in the 
scope of the project. But for example reading a file of e.g. less than 5 kB 
should allways be that quick that I would consider the effort of asynch 
communication as too much.
If this takes long you have severe problems with your infrastructure which are 
worth while to be checked and solved.

Original comment by cor...@endt.info on 5 Jul 2011 at 9:45

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi,

Let me rephrase requirement. 

I am not suggesting to throw away the original web services which are 
synchronous.

I am suggesting to add extra web services to support async operations.

Regarding file size:

I assume that File Connector is supposed to work on polling files, reading 
structured files, etc.,

In a typical production environment ( not a poc environment), there will be a 
definition of boundary limits on file size (both min size and max size). 

One should not always plan for min size. Any component should think about 
handling maximum file size or at least define maximum file size for typical 
disk IO.

If the requirement defines that the file size is always small for a project, 
then the implementation could be based on synchronous operations.

If the requirement defines that the file size can vary between min and max, 
then the implementation should be based on asynchronous operations wherever 
suitable.

Also, I am not suggesting that it is mandatory to define asynchronous web 
services for File Connector. One can always write their own connector if a 
project requires to deal with huge files.

The suggested enhancement is only a value addition.

Original comment by udayku...@gmail.com on 6 Jul 2011 at 9:21