earlng
commented
3 years ago 285baacbdf8fda1de94b19282acd23e2
The reason the impact statement for this one doesn't show up anymore is because immediately following the "Broader Impact" header the statement is in an h2 class that goes into detail about the statement itself.
<section class="DoCO:Section">
<h1 class="DoCO:SectionTitle" id="145" page="10" column="1">Broader Impact</h1>
<section class="DoCO:Section">
<h2 class="DoCO:SectionTitle" id="146" confidence="possible" page="10" column="1">Exploring Memory-Computation Trade-offs in RL</h2>
<region class="DoCO:TextChunk" id="151" page="10" column="1">Reinforcement learning policies have enjoyed remarkable success in recent years, in particular in the context of large-scale game playing. These results, however, mask the high underlying costs in terms of computational resources and training time that the demonstrations requires [<xref ref-type="bibr" rid="R36" id="147" class="deo:Reference">36</xref>, <xref ref-type="bibr" rid="R26" id="148" class="deo:Reference">26</xref>, <xref ref-type="bibr" rid="R27" id="149" class="deo:Reference">27</xref>, <xref ref-type="bibr" rid="R35" id="150" class="deo:Reference">35</xref>]. For example, the AlphaGo Zero algorithm that mastered Chess and Go from scratch trained their algorithm over 72 hours using 4 TPUs and 64 GPUs. These results, while highlighting the intrinsic power in reinforcement learning algorithms, are computationally infeasible for applying algorithms to RL tasks in computing systems. As an example, RL approaches have received much interest in several of the following problems:</region>
<region class="DoCO:TextChunk" id="159" confidence="possible" page="10" column="1">• Memory Management : Many computing systems have two sources of memory; on-chip memory which is fast but limited, and off-chip memory which has low bandwidth and suffers from high latency. Designing memory controllers for these system require a scheduling policy to adapt to changes in workload and memory reference streams, ensuring consistency in the memory, and controlling for long-term consequences of scheduling decisions [<xref ref-type="bibr" rid="R1" id="152" class="deo:Reference">1</xref>, <xref ref-type="bibr" rid="R2" id="153" class="deo:Reference">2</xref>, <xref ref-type="bibr" rid="R8" id="154" class="deo:Reference">8</xref>]. • Online Resource Allocation : Cloud-based clusters for high performance computing must decide how to allocate computing resources to different users or tasks with highly variable demand. Controllers for these systems must make decisions online to manage the trade-offs between computation cost, server costs, and delay in job-completions. Recent work has studied RL algorithms for such problems [<xref ref-type="bibr" rid="R15" id="155" class="deo:Reference">15</xref>, <xref ref-type="bibr" rid="R23" id="156" class="deo:Reference">23</xref>, <xref ref-type="bibr" rid="R28" id="157" class="deo:Reference">28</xref>, <xref ref-type="bibr" rid="R22" id="158" class="deo:Reference">22</xref>].</region>
<region class="DoCO:TextChunk" id="160" page="10" column="1">Common to all of these examples are computation and storage limitations on the devices used for the controller.</region>
<region class="DoCO:TextChunk" id="161" confidence="possible" page="10" column="1">• Limited Memory : On chip memory is expensive and off-chip memory access has low- bandwidth. As any reinforcement learning algorithm requires memory to store estimates of relevant quantities - RL algorithms for computing systems must manage their computational requirements. • Power Consumption : Many applications require low-power consumption for executing RL policies on general computing platforms. • Latency Requirements : Many problems for computing systems (e.g. memory management) have strict latency quality of service requirements that limits reinforcement learning algorithms to execute their policy quickly.</region>
<region class="DoCO:TextChunk" id="167" page="10" column="1">Our algorithm A DA MB takes a first step towards designing efficient reinforcement learning algorithms for continuous (or large finite) spaces, where efficient means both low-regret, but also low storage and computation complexity (see <xref ref-type="table" rid="T1" id="162" class="deo:Reference">Table 1</xref>). A DA MB is motivated by recent algorithms for reinforcement learning on memory constrained devices which use a technique called cerebellar model articulation controller (CMAC). This technique uses a random-discretizations of the space at various levels of coarseness [<xref ref-type="bibr" rid="R15" id="163" class="deo:Reference">15</xref>]. Moreover, heuristic algorithms which use discretizations (either fixed or adaptive) have been extensively studied on various tasks [<xref ref-type="bibr" rid="R32" id="164" class="deo:Reference">32</xref>, <xref ref-type="bibr" rid="R39" id="165" class="deo:Reference">39</xref>, <xref ref-type="bibr" rid="R22" id="166" class="deo:Reference">22</xref>]. We are able to show that our algorithm achieves good dependence with respect to K on all three dimensions (regret, computation, and storage complexity). With future work we hope to determine problem specific guarantees, exhibiting how these adaptive partitioning algorithms are able to extract structure common in computing systems problems.</region>
<outsider class="DoCO:TextBox" type="page_nr" id="168" page="10" column="1">10</outsider>
</section>cdfa4c42f465a5a66871587c69fcfa34 seems to suffer the same issue.
as does 33a854e247155d590883b93bca53848a although what follows isn't immediately an h2 header.
<section class="DoCO:Section">
<h1 class="DoCO:SectionTitle" id="185" page="10" column="1">Broader Impact</h1>
<region class="unknown" id="186" page="10" column="1">Who may benefit from this research</region>
<region class="DoCO:TextChunk" id="192" page="10" column="1">Our research presumably has quite broad impact, since discovery of mathematical patterns in data is a central problem across the natural and social sciences. Given the ubiquity of linear regression in research, one might expect that there will significant benefits to a broad range of researchers also from more general symbolic regression once freely available algorithms get sufficiently good. <marker type="block"/> Although it is possible that some numerical modelers could get their jobs automated away by symbolic regression, we suspect that the main effect of our method, and future tools building on it, will instead be that these people will simply discover better models than today.<marker type="block"/> Pareto-optimal symbolic regression can be viewed as an extreme form of lossy data compression that uncovers the simplest possible model for any given accuracy. To the extent that overfitting can exacerbate bias, such model compression is expected to help. Moreover, since our method produces closed-form mathematical formulas that have excellent interpretability compared to black-box neural networks, they make it easier for humans to interpret the computation and pass judgement on whether it embodies unacceptable bias. This interpretability also reduces failure risk. Another risk is automation bias, whereby people overly trust a formula from symbolic regression when they extrapolate it into an untested domain. This could be exacerbated if symbolic regression promotes scientific laziness and enfeeblement, where researchers fit phenomenological models instead of doing the work of building models based on first principles. Symbolic regression should inform but not replace traditional scientific discovery. Although the choice of basis functions biases the discoverable function class, our method is agnostic to basis functions as long as they are mostly differentiable. The greatest potential risk associated with this work does not stem from it failing but from it suc- ceeding: accelerated progress in symbolic regression, modularity discovery and its parent discipline, program synthesis, could hasten the arrival of artificial general intelligence, which some authors have argued humanity still lacks the tools to manage safely [<xref ref-type="bibr" rid="R5" id="191" class="deo:Reference">5</xref>]. On the other hand, our work may help accelerate research on intelligible intelligence more broadly, and powerful future artificial intelligence is probably safer if we understand aspects of how it works than if it is an inscrutable black box.</region>
<region class="unknown" id="188" page="10" column="1">Who may be put at disadvantage from this research</region>
<region class="unknown" id="190" page="10" column="1">Risk of bias, failure and other negative outcomes</region>
<outsider class="DoCO:TextBox" type="page_nr" id="193" page="10" column="1">10</outsider>
</section>Seen again in 4496bf24afe7fab6f046bf4923da8de6
<section class="DoCO:Section">
<h1 class="DoCO:SectionTitle" id="200" page="10" column="1">Broader Impact</h1>
<region class="unknown" id="201" page="10" column="1">Positive impact</region>
<region class="DoCO:TextChunk" id="207" page="10" column="1">Our work provides a solution to learn a policy that generalizes to a set of similar tasks from only observational data. The techniques we propose have great potential to benefit various areas of the whole society. For example in the field of healthcare, we hope the proposed triplet loss design with hard negative mining can enable us to robustly train an automatic medical prescription system from a large batch of medical histories of different diseases and further generalize to new diseases [ <xref ref-type="bibr" rid="R61" id="202" class="deo:Reference">61</xref>], e.g., COVID-19. Moreover, in the field of robotics, our methods can enable the learning of a single policy that solves a set of similar unseen tasks from only historical robot experiences, which tackles the sample efficiency issues given that sampling is expensive in the field of real-world robotics [<xref ref-type="bibr" rid="R46" id="203" class="deo:Reference">46</xref>]. Even though in some fields that require safe action selections, e.g, autonomous driving [<xref ref-type="bibr" rid="R62" id="204" class="deo:Reference">62</xref>] and medical prescription, our learned policy cannot be immediately applied, it can still serve as a good prior to accelerate further training.<marker type="block"/> Evidently, the algorithm we proposed is a data-driven methods. Therefore, it is very likely that it will be biased by the training data. Therefore, if the testing tasks are very different from the training tasks, the learned policy may even result in worse behaviors than random policy, leading to safety issues. This will motivate research into safe action selection and distributional shift identification when learning policies for sequential process from only observational data.</region>
<region class="unknown" id="206" page="10" column="1">Negative impact</region>
<outsider class="DoCO:TextBox" type="page_nr" id="208" page="10" column="1">10</outsider>
</section>
Examples of BIS that are not being pulled under the new code but used to be: