azist / azos

A to Z Sky Operating System / Microservice Chassis Framework
MIT License
213 stars 29 forks source link

Add `Azos.Sky.FileGateway` needed to upload files across cloud/onprem boundaries #883

Closed itadapter closed 1 year ago

itadapter commented 1 year ago

Provides a service to external callers to download and upload files on a local disk/network using designated internal disk/network mapping.

The use of gateway obviates the need to use external services such as SCP/FTPS etc. as it allows to perform similar actions via API (such as REST) interface via well-known endpoints

itadapter commented 1 year ago

Issues identified during test session today:

  1. Item list fails if there are no files (empty directory)
  2. Item list fails on empty root path, e,g, "bom@mfg-in::/"
  3. Create an empty dir, and when item;list is called 591 is returned inner excpetion "Could not find file.....\aaa' where aaa is dir name; does not matter if dir is empty
  4. Create directory, similar error 591 - can not find file .get_Length()
  5. item get does not work for dir: 591 Could not find file; for a 1 file it works
  6. Download file pads output up to SIZE parameter like: data:"jfjdsfkjdlkAAAAAAAAAAAAAAAAAAAAAAAAAAA"
  7. 591 on file delete that was deleted

Separately: If I pass wrong say volume, I get 400 bad request, but MvcActionException does not provide "data" from internal FilterPipelineException-> GuardedMethodException - this needs research. Notice that exception details are missing in "data"

{
  "OK": false, 
  "http::code": 400, 
  "http::descr": "Bad Request", 
  "error": "MvcActionException", 
  "isauth": false, 
  "data": 
    {
      "ns": "wave.mvc", 
      "type": "MvcActionException", 
      "code": 0, 
      "controller": "Azos.Sky.FileGateway.Server.Web.Gateway", 
      "action": "file"
    }, 
  "dev::dump": 
    {
      "TypeName": "Azos.Wave.FilterPipelineException", 
      "Message": ">>error>realip>session>dconv>latency>  [Azos.Wave.MvcActionException] Controller action body: 'Azos.Sky.FileGateway.Server.Web.Gateway'.'file'. Exception: [Azos.CallGuardException] Guarded method 'mailto:item@gatewaysystem.cs:69' item 'Gateway system `mfg-in`, volume `bxxx`' is not found", 
      "Code": 0, 
      "Source": "Azos.Wave", 
      "StackTrace": "   at Azos.Wave.WorkFilter.FilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.InvokeNextWorkerAsync(WorkContext work, CallChain callChain)\r\n   at G8.Server.Web.DataConversionFilter.DoFilterWorkAsync(WorkContext work, CallChain callChain) in C:\\dev\\g8\\src\\G8.Server\\Web\\DataConversionFilter.cs:line 67\r\n   at Azos.Wave.WorkFilter.FilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.InvokeNextWorkerAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.Filters.SessionFilter.DoFilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.FilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.InvokeNextWorkerAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.Filters.EffectiveCallerIPEndPointFilter.DoFilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.FilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.InvokeNextWorkerAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.Filters.ErrorFilter.DoFilterWorkAsync(WorkContext work, CallChain callChain)", 
      "AppId": "nlink", 
      "AppName": "nlink", 
      "ExternalStatus": null, 
      "InnerException": 
        {
          "TypeName": "Azos.Wave.MvcActionException", 
          "Message": "Controller action body: 'Azos.Sky.FileGateway.Server.Web.Gateway'.'file'. Exception: [Azos.CallGuardException] Guarded method 'mailto:item@gatewaysystem.cs:69' item 'Gateway system `mfg-in`, volume `bxxx`' is not found", 
          "Code": 0, 
          "Source": "Azos.Wave", 
          "StackTrace": "   at Azos.Wave.Handlers.MvcHandler.DoError(WorkContext work, Exception error)\r\n   at Azos.Wave.Handlers.TypeLookupHandler`1.DoHandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.WorkHandler.HandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.WorkHandler.FilterAndHandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.CompositeWorkHandler.DoHandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.WorkHandler.HandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.WorkHandler.FilterAndHandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.CompositeWorkHandler.DoHandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.WorkHandler.HandleWorkAsync(WorkContext work)\r\n   at Azos.Wave.WorkFilter.InvokeNextWorkerAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.Filters.LatencyInstrumentationFilter.DoFilterWorkAsync(WorkContext work, CallChain callChain)\r\n   at Azos.Wave.WorkFilter.FilterWorkAsync(WorkContext work, CallChain callChain)", 
          "AppId": "nlink", 
          "AppName": "nlink", 
          "ExternalStatus": null, 
          "InnerException": 
            {
              "TypeName": "Azos.CallGuardException", 
              "Message": "Guarded method 'mailto:item@gatewaysystem.cs:69' item 'Gateway system `mfg-in`, volume `bxxx`' is not found", 
              "Code": 0, 
              "Source": "Azos.Sky.Server", 
              "StackTrace": "   at Azos.Sky.FileGateway.Server.GatewaySystem.get_Item(Atom volume)\r\n   at Azos.Sky.FileGateway.Server.DefaultFileGatewayLogic.getVolume(EntityId path)\r\n   at Azos.Sky.FileGateway.Server.DefaultFileGatewayLogic.CreateFileAsync(EntityId path, CreateMode mode, Int64 offset, Byte[] content)\r\n   at Azos.Sky.FileGateway.Server.Web.Gateway.CreateFile(EntityId path, CreateMode mode, Int64 offset, Byte[] content)\r\n   at Azos.Wave.Handlers.MvcHandler.DoTargetWorkAsync(Controller target, WorkContext work)\r\n   at Azos.Wave.Handlers.MvcHandler.DoTargetWorkAsync(Controller target, WorkContext work)", 
              "AppId": "nlink", 
              "AppName": "nlink", 
              "ExternalStatus": null, 
              "InnerException": null
            }
        }
    }
}
itadapter commented 1 year ago
itadapter commented 1 year ago

FileGatewayError should not set Http status code if it has inner exception.