search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

API Doc Improvements - Tracking #6000

Closed domdivakaruni closed 7 years ago

domdivakaruni commented 7 years ago

Let's fix our docs!

The effort to spruce up MXNet's Python API docs is picking up steam but could use your help to add that extra oomph! Many of the APIs now have proper explanations and good examples, but we still have much more to go!

alt text

Here’s how you can join in and help:

  1. Pick a few functions (or at least 1 :smile: ) from the list below

    • Ensure you only pick ones that don’t have a current owner assigned and you can submit within 2 business days. Post a comment to the issue with the ones you’ve selected.

  2. Look up current documentation

  3. Write up:

    • A clear and concise description of the function and its behavior.
    • List and describe each parameter with the valid input values, whether it is required or optional, and default value if the parameter is optional.
    • Illustrate the function and parameter behavior using examples
    • Refer to these examples for guidance:- Embedding , ROIPooling , Reshape

  4. Submit your work

    • Creating a new issue with [API docs submission] ‘your function name(s)’ in the title and paste your write-up markdown OR
    • Creating a pull request for a change to the appropriate .cc or .py file. Instructions here: How to make a pull request for docs

API List

# Function Name Package Contributor Status
1
  • - [ ] IRHeader
recordio
2
  • - [ ] pack
recordio
3
  • - [ ] pack_img
recordio
4
  • - [ ] unpack
recordio
5
  • - [ ] unpack_img
recordio
6
  • - [ ] BilinearSampler
ndarray @Lyken17
7
  • - [ ] Correlation
ndarray @Lyken17
8
  • - [ ] GridGenerator
ndarray @Lyken17
9
  • - [ ] IdentityAttachKLSparseReg
ndarray @Lyken17
10
  • - [ ] SpatialTransformer
ndarray @Lyken17
11
  • - [ ] UpSampling
ndarray @Lyken17
12
  • - [ ] adam_update
ndarray
13
  • - [ ] ones_like
ndarray @yuruofeifei
14
  • - [ ] random_exponential
ndarray @asmushetzel
15
  • - [ ] random_gamma
ndarray @asmushetzel
16
  • - [ ] random_generalized_negative_binomial
ndarray @asmushetzel
17
  • - [ ] random_negative_binomial
ndarray @asmushetzel
18
  • - [ ] random_poisson
ndarray @asmushetzel
19
  • - [x] rmsprop_update
ndarray @jiajiechen
20
  • - [x] rmspropalex_update
ndarray @jiajiechen
21
  • - [ ] sample_exponential
ndarray @asmushetzel
22
  • - [ ] sample_gamma
ndarray @asmushetzel
23
  • - [ ] sample_generalized_negative_binomial
ndarray @asmushetzel
24
  • - [ ] sample_negative_binomial
ndarray @asmushetzel
25
  • - [ ] sample_normal
ndarray @asmushetzel
26
  • - [ ] sample_poisson
ndarray @asmushetzel
27
  • - [ ] sample_uniform
ndarray @asmushetzel
28
  • - [ ] sgd_mom_update
ndarray
29
  • - [ ] sgd_update
ndarray
30
  • - [ ] smooth_l1
ndarray
31
  • - [ ] softmax_cross_entropy
ndarray @yuruofeifei
32
  • - [ ] waitall
ndarray
33
  • - [ ] zeros_like
ndarray @yuruofeifei
34
  • - [ ] Caffe
metric
35
  • - [ ] Torch
metric
36
  • - [ ] check_label_shapes
metric
37
  • - [ ] Bilinear
initializer
38
  • - [ ] FusedRNN
initializer
39
  • - [ ] LSTMBias
initializer
40
  • - [ ] LogValidationMetricsCallback
callback
41
  • - [ ] _init_torch_module
torch
42
  • - [ ] _make_torch_function
torch
43
  • - [ ] CustomOp.assign
operator
44
  • - [ ] NameManager
name
45
  • - [ ] Monitor
monitor
46
  • - [ ] CastAug
image
47
  • - [ ] CenterCropAug
image
48
  • - [ ] ColorJitterAug
image
49
  • - [ ] ColorNormalizeAug
image
50
  • - [ ] CreateAugmenter
image
51
  • - [ ] HorizontalFlipAug
image
52
  • - [ ] LightingAug
image
53
  • - [ ] RandomCropAug
image
54
  • - [ ] RandomOrderAug
image
55
  • - [ ] RandomSizedCropAug
image
56
  • - [ ] ResizeAug
image
57
  • - [ ] center_crop
image
58
  • - [ ] color_normalize
image
59
  • - [ ] fixed_crop
image
60
  • - [ ] imdecode
image
61
  • - [ ] random_crop
image
62
  • - [ ] random_size_crop
image
63
  • - [ ] resize_short
image @jhwei
64
  • - [ ] scale_down
image @jhwei
65
  • - [ ] ImageIter.augmentation_transform
image
66
  • - [ ] ImageIter.imdecode
image
67
  • - [ ] ImageIter.postprocess_data
image
68
  • - [ ] ImageIter.read_image
image
69
  • - [ ] ImageIter.check_data_shape
image
70
  • - [ ] ImageIter.check_valid_image
image
71
  • - [ ] Executor._get_dict
executor
72
  • - [ ] _monitor_callback_wrapper
executor
73
  • - [ ] MultiBoxDetection
contrib
74
  • - [ ] MultiBoxPrior
contrib
75
  • - [ ] MultiBoxTarget
contrib
76
  • - [ ] Proposal
contrib
77
  • - [ ] count_sketch
contrib
78
  • - [ ] fft
contrib
79
  • - [ ] ifft
contrib
80
  • - [ ] set_is_training
contrib
81
  • - [ ] test
contrib
82
  • - [ ] train
contrib
83
  • - [ ] MXNetError
base
84
  • - [ ] mx_float_p
base
85
  • - [ ] py_str
base
86
  • - [ ] print_summary
visualization
87
  • - [ ] Symbol.attr
symbol
88
  • - [ ] Symbol.attr_dict
symbol
89
  • - [ ] Symbol.debug_str
symbol
90
  • - [ ] Symbol.grad
symbol
91
  • - [ ] Symbol.list_attr
symbol
92
  • - [ ] MXRecordIO.close
recordio
93
  • - [ ] MXRecordIO.open
recordio
94
  • - [ ] Optimizer.set_lr_mult
optimizer
95
  • - [ ] Optimizer.set_lr_scale
optimizer
96
  • - [ ] Optimizer.set_wd_mult
optimizer
97
  • - [ ] Optimizer.update
optimizer
98
  • - [ ] LinearRegressionOutput
ndarray @naidu0830
99
  • - [x] NDArray._at
ndarray @reminisce
100
  • - [x] NDArray._slice
ndarray @reminisce
101
  • - [x] NDArray._sync_copyfrom
ndarray @reminisce
102
  • - [ ] gamma
ndarray @zhenwendai
103
  • - [ ] gammaln
ndarray @zhenwendai
104
  • - [ ] identity
ndarray @zhenwendai
105
  • - [ ] load
ndarray @zhenwendai
106
  • - [ ] moveaxis
ndarray @zhenwendai
107
  • - [ ] DataDesc
io
108
  • - [ ] DataDesc.get_batch_axis
io
109
  • - [ ] DataDesc.get_list
io
110
  • - [ ] DataIter.getpad
io
111
  • - [ ] MXDataIter
io
112
  • - [ ] ImageDetRecordIter
io
113
  • - [ ] Initializer._legacy_init
initializer
114
  • - [ ] Initializer.dumps
initializer
115
  • - [ ] ProgressBar
callback @saurabh3949
116
  • - [ ] log_train_metric
callback
117
  • - [ ] module_checkpoint
callback
118
  • - [ ] _parse_aux_states
test_utils
119
  • - [ ] _parse_location
test_utils
120
  • - [x] check_symbolic_backward
test_utils @reminisce
121
  • - [x] check_symbolic_forward
test_utils @reminisce
122
  • - [ ] np_reduce
test_utils
123
  • - [ ] numeric_grad
test_utils
124
  • - [ ] print_max_err_loc
test_utils
125
  • - [ ] BucketSentenceIter
rnn
126
  • - [ ] encode_sentences
rnn
127
  • - [ ] do_rnn_checkpoint
rnn
128
  • - [ ] load_rnn_checkpoint
rnn
129
  • - [ ] save_rnn_checkpoint
rnn
130
  • - [ ] BaseRNNCell.begin_state
rnn
131
  • - [ ] BaseRNNCell.pack_weights
rnn
132
  • - [ ] BaseRNNCell.unpack_weights
rnn
133
  • - [ ] BaseRNNCell.unroll
rnn
134
  • - [ ] BidirectionalCell
rnn
135
  • - [ ] DropoutCell
rnn
136
  • - [ ] FusedRNNCell
rnn
137
  • - [ ] GRUCell
rnn
138
  • - [ ] LSTMCell
rnn
139
  • - [ ] ModifierCell
rnn
140
  • - [ ] RNNCell
rnn
141
  • - [ ] RNNParams
rnn
142
  • - [ ] RNNParams.get
rnn
143
  • - [ ] SequentialRNNCell
rnn
144
  • - [ ] SequentialRNNCell.add
rnn
145
  • - [ ] ZoneoutCell
rnn
146
  • - [ ] BaseRNNCell._get_activation
rnn
147
  • - [ ] FusedRNNCell._slice_weights
rnn
148
  • - [ ] dump_profile
profiler
149
  • - [ ] profiler_set_config
profiler
150
  • - [ ] profiler_set_state
profiler
151
  • - [ ] CustomOp
operator
152
  • - [ ] CustomOp.backward
operator
153
  • - [ ] CustomOp.forward
operator
154
  • - [ ] CustomOpProp
operator
155
  • - [ ] CustomOpProp.infer_shape
operator
156
  • - [ ] CustomOpProp.list_arguments
operator
157
  • - [ ] CustomOpProp.list_auxiliary_states
operator
158
  • - [ ] CustomOpProp.list_outputs
operator
159
  • - [x] NDArrayOp.declarebackward
operator
160
  • - [x] NumpyOp
operator
161
  • - [x] PythonOp
operator
162
  • - [x] PythonOp.backward
operator
163
  • - [x] PythonOp.forward
operator
164
  • - [x] PythonOp.get_symbol
operator
165
  • - [x] PythonOp.infer_shape
operator
166
  • - [x] PythonOp.list_arguments
operator
167
  • - [x] PythonOp.list_outputs
operator
168
  • - [x] PythonOp.need_top_grad
operator
169
  • - [ ] register
operator
170
  • - [ ] CustomOpProp.infer_type
operator
171
  • - [ ] NameManager.get
name
172
  • - [ ] Prefix
name
173
  • - [ ] Monitor.install
monitor
174
  • - [ ] Monitor.tic
monitor
175
  • - [ ] Monitor.toc
monitor
176
  • - [ ] Monitor.toc_print
monitor
177
  • - [ ] getLogger
log
178
  • - [x] KVStore.load_optimizer_states
kvstore @eric-haibin-lin
179
  • - [x] KVStore.save_optimizer_states
kvstore @eric-haibin-lin
180
  • - [x ] KVStore.set_optimizer
kvstore @eric-haibin-lin
181
  • - [x] create
kvstore @eric-haibin-lin
182
  • - [ ] ImageIter
image
183
  • - [ ] ImageIter.next
image
184
  • - [ ] ImageIter.next_sample
image
185
  • - [ ] ImageIter.reset
image
186
  • - [x] Executor
executor @kevinthesun
187
  • - [x] Executor.backward
executor @kevinthesun
188
  • - [x] Executor.copy_params_from
executor @kevinthesun
189
  • - [x] Executor.reshape
executor @kevinthesun
190
  • - [x] Executor.set_monitor_callback
executor @kevinthesun
191
  • - [ ] Executor._get_outputs
executor
192
  • - [x] Executor.debug_str
executor @kevinthesun
193
  • - [ ] compute_gradient
contrib
194
  • - [ ] grad_and_loss
contrib
195
  • - [ ] mark_variables
contrib
196
  • - [ ] set_recording
contrib
197
  • - [ ] LogMetricsCallback
contrib
198
  • - [ ] TrainingStateScope
contrib
199
  • - [ ] build_param_doc
base
200
  • - [ ] c_array
base @yuruofeifei
201
  • - [ ] c_str
base @yuruofeifei
202
  • - [ ] check_call
base
203
  • - [ ] ctypes2buffer
base
204
  • - [ ] ctypes2numpy_shared
base
205
  • - [ ] _load_lib
base
206
  • - [ ] _notify_shutdown
base
207
  • - [ ] add_fileline_to_docstring
base
208
  • - [ ] AttrScope
attribute
209
  • - [ ] AttrScope.current
attribute
210
  • - [ ] AttrScope.get
attribute
piiswrong commented 7 years ago

NumpyOP pythonop and ndarrayop are deprecated. We only support customop now

rongali-naidu commented 7 years ago

I will document "LinearRegressionOutput" API

asmushetzel commented 7 years ago

I will document all samplers and randoms as I anyway enhanced or authored them, namely

random_gamma random_exponential random_poisson random_negative_binomial random_generalized_negative_binomial

sample_uniform sample_normal sample_gamma sample_exponential sample_poisson sample_negative_binomial sample_generalized_negative_binomial

I may need more than 2 business days but should be definitely done by end of next week.

moontails commented 7 years ago

I will document Functions from 125 to 147 in the rnn package.

If I get more time, I will also work on 161-168

domdivakaruni commented 7 years ago

@moontails that is tremendous thank you! Do you plan on making PR's or creating Issues with your functions in markdown? When you are done with your first function send it in so we can help with a review before you get rolling with the rest. great stuff!

moontails commented 7 years ago

@domdivakaruni I was thinking of creating PR's.

Thats a good idea! I will submit a PR when am done with the changes for the first function and gather all feedback before diving in on the rest :)

zhenwendai commented 7 years ago

I can document some of ndarray operators:

102 gamma ndarray
103 gammaln ndarray
104 identity ndarray
105 load ndarray
106 moveaxis ndarray

domdivakaruni commented 7 years ago

Hi @moontails do you want to call ownership of all 22 rnn functions at once or do you want to stake your claim in batches as you make progress?

yuruofeifei commented 7 years ago

I will work on zeros_like, ones_like, softmax_cross_entropy, c_array, c_str

jhwei commented 7 years ago

I can work on resize_short, scale_down.

domdivakaruni commented 7 years ago

@yuruofeifei @jhwei @zhenwendai good stuff!

moontails commented 7 years ago

@domdivakaruni either works fine for me. I had hoped to send out a PR yesterday to get some feedback before working on the rest, but ran into some issues setting up the repo (with the whole installing mxnet and make lint)

domdivakaruni commented 7 years ago

@moontails did you get it worked out? Let me know what you ran into. @nswamy fyi

nswamy commented 7 years ago

@moontails whats the issue that you are running into? @domdivakaruni can we have folks picks few APIs at once and add more to their bucket as they make progress?

Lyken17 commented 7 years ago

I can document 6~11.

Question : Should we use reST or Markdown for improvement through github issues? This page says Markdown, but pull request pages requires reST.

nswamy commented 7 years ago

@Lyken17 please prefer to use reST and submit a PR. If you want to just submit doc outside of the the code(python doc string or describe method of C++ code) use either reST or Markdown and submit an issue.

nswamy commented 7 years ago

@zhenwendai, The docs for these APIs are already updated. Could you please pick a different set? 102 gamma ndarray 103 gammaln ndarray 104 identity ndarray

srviest commented 7 years ago

I will work on 58 color_normalize. BTW, where are the definitions of 78 fft & 79 ifft?

jiayue666 commented 7 years ago

I can document 2-5.

szha commented 7 years ago

I can share 130-145 with @moontails. Let me know which ones would you prefer that I do.

@piiswrong do we need to document 146 BaseRNNCell._get_activation and 147 FusedRNNCell._slice_weights? These are not intended for use other than by the APIs.

domdivakaruni commented 7 years ago

@moontails are you still working on these APIs? What can you share with @szha? can you please let us know as soon as you can? Thanks!

moontails commented 7 years ago

@domdivakaruni @szha I am sorry guys, I still havent been able to get it to work on my Macbook.

I will let @szha start checking things off from 130-145 and get back on track if I am able to unblock myself.

asmushetzel commented 7 years ago

@moontails Don't know which problem you ran into but also had to resolve some hiccups on my mac. Resolved it by:

That got me going. I used a separate mxnet repo for standard building/installation and functional testing (I had to touch a bit more than just the "describe" section of the operators). Don't know whether this was necessary, but at least it worked that way.

Did punt on lint on my mac as I couldn't get it to work. Rather did submit the pull request and fixed the flagged lint issues based on the automatic tests. It's straightforward.

eric-haibin-lin commented 7 years ago

I'm going to take the four APIs for kvstore 178 - 181

nswamy commented 7 years ago

@Lyken17 any luck updating the doc for the APIs that you picked ?

Lyken17 commented 7 years ago

Hi @nswamy

I just updated GridGenerator and submit it to issue https://github.com/dmlc/mxnet/issues/6147.

I meet troubles when documenting ndarray.UpSampling. Even myself cannot find the proper way to use it, according to issues https://github.com/dmlc/mxnet/issues/2823 https://github.com/dmlc/mxnet/issues/4134 https://github.com/dmlc/mxnet/issues/1412, it seems to be an incomplete implementation. I also looked up several FCN mxnet implementations, they all use deconvolution instead of UpSampling.

nswamy commented 7 years ago

@Lyken17, Thanks for posting the issue, I will convert this to actual documentation in code. You can skip UpSampling for now, we might need to add a note that it is incomplete. @mli, @piiswrong what do you suggest?

For other APIs that you work on, can you please edit the docs in the code directly? If building and rendering takes time or you get into an issue, you can skip that step and just follow the tips to avoid any lint errors.

apaleyes commented 7 years ago

Hi @nswamy

I've just posted a pr for 118 and 119: https://github.com/dmlc/mxnet/pull/6314

One thing though is that I wasn't able to get rendering going on my Mac. "make lint" was fine, but building docs was always failing. I'll post a separate issue perhaps to get some advice on that.