Add support for DELETE, PUT, POST, MERGE operation

[phanak-sap]

placeholder issue , will be specified in more detail

Library / DirectBuilder usage = experimental prototype for this feature.

• [x] @AlapanGhosh - add support for generating GET and DELETE url parts for entities for DirectBuilder usage only.
  • PR #72
• [x] @AlapanGhosh - add support for generating PUT - also only url part
  • PR #81
• [x] @AlapanGhosh - add support for generating POST - solve http body payload fuzzing
  • PR #81
• [x] @AlapanGhosh - add support for generating MERGE - solve http body payload fuzzing
  • PR #91
• [x] @AlapanGhosh - developer (library usage, DirectBuilder) documentation - https://github.com/SAP/odfuzz/blob/master/doc/http_methods.rst

CLI USAGE

• [ ] @phanak-sap - refactor fuzzer.py, Query class and most importantly the def build_string() usage for different HTTP methods.

  • Here clashes code flow of normal CLI usage and DirectBuilder usage (going from "below" from entities.py); also clash of configuration by env. variables and Config class (CLI usage) versus constructors/functions parameters configuration in DirectBuilder usage as python library.
  • Query class also does not ONLY cover bulding an url part, but also has properties relevant to fuzzing process itself (generation+mutation) and is also closely tied with specific database (MongoDB, line 1031 self._id = ObjectId() ), making the necessary separation of fuzzer.py module into smaller and more isolated modules more complicated.
  • Dispatcher itself expects only sending GET HTTP requests at the moment.

• [ ] @phanak-sap - add support for GET and DELETE for CLI usage of odfuzz also.

• [ ] @phanak-sap - bump versionfor this new feature

[prashdsouza]

Delete queries with filter option

https://stackoverflow.com/questions/33279744/odata-delete-with-filter

[AlapanGhosh]

Hi, I looked through the v2 operations documentation. Based on sections 2.8 and 2.11, and also the above attached stackoverflow thread, we can assume DELETE wont be taking any query options, other than while deleting links.

Which would mean the first step would be to decouple the invocation of the query-options generation in the workflow. Any pointers on where and how I can safely stop the query-options generation in the current GET workflow? Thanks

[lubosmj]

@AlapanGhosh, I believe you have to modify the following method: https://github.com/SAP/odfuzz/blob/41d244111b0255854bcff3ecbc68bf679a59312c/odfuzz/fuzzer.py#L353-L361

Query options are generated and added to a specific query in the for loop. The method build_string() assembles a final query string from an entity's URI and query options' values: https://github.com/SAP/odfuzz/blob/41d244111b0255854bcff3ecbc68bf679a59312c/odfuzz/fuzzer.py#L1115-L1133

As long as a query does not reference any query options, the for loop will not be executed.

Also, I recommend you to take a look at the method _add_appendix() which is called from the method build_string(). This method appends additional query options to a URL by default: https://github.com/SAP/odfuzz/blob/41d244111b0255854bcff3ecbc68bf679a59312c/odfuzz/fuzzer.py#L1164-L1168

[AlapanGhosh]

@lubosmj Thanks for the pointers. It was really helpful.

[AlapanGhosh]

Hi @phanak-sap

As per our last discussion you mentioned you have not committed the n-odata-server test application. So, would you be in a position to commit so that I can install the test app locally and test out the DELETE PoC? Thanks

[phanak-sap]

@AlapanGhosh I only tested that the n-odata-server (from samples) produces valid metadata, so it is not stopped on first step. Then I went to vacation as discussed. So at the moment I don't have the sample app (but I guess you can use the internal fiorigoat for testing first changes)

Also, I am against direct modifying the generate_options() method in the fuzzer.py. It would be better, to as part of module de-coupling, to move anything relevant to URL query generation - this method and accompanying utilities - out of fuzzer.py to most probably new module or at least new class in entities.py and just call the respective URL generation (GET/DELETE) from fuzzer.py.

Remember, it would be good to move towards like a library code, that would generate URLs (and after that, mutates them) for provided metadata, without networking and database and CLI / gevent runtime.

[phanak-sap]

I am closing this issue with no CLI task implemented, just covering the support in the DirectBuilder only.

There is currently discussed variant that odfuzz will in the future use only the DirectBuilder in some core module, to better modularize the project. DispatchedBuilder entanglement with practically all other parts of odfuzz (e.g. in issue description) make it really hard to maintain. In the future there is also possibility to remove mongoDB dependency (and perhaps even drop the mutation part), in favor of more CLI usage with sqlite/files only.