Logemann, Torben (2023) Explainability of power grid attack strategies learned by Deep Reinforcement Learning Agents. Masters, University of Oldenburg.

Preview

Volltext (3447Kb)

Abstract

Learning systems have achieved remarkable success. Agents trained using Deep RL (DRL) methods, e.g., promise true resilience. However, no guarantees can yet be given for the black-box models that have been learned. For operators of Critical National Infrastructures (CNIs), this is a necessity, as no responsibility for an unknown and unverifiable control system can be assumed. Intrinsically safe learning algorithms and approximate, post-hoc interpretable models exist, but they lack either learning power or explainability. To optimize this trade-off, this thesis presents the NN2EQCDT algorithm, which directly converts a (whole) policy- based Feed-Forward DNN (FF-DNN) into a compressed Decision Tree (DT). Compression is achieved by dynamically checking the satisfiability of the paths of the transformed DTs during construction and disregarding unneeded rules. It can be further increased by using additional methods and considering invariants. It has been observed that the NN2EQCDT algorithm can drastically compress a small policy model, making it possible to exactly track the action regions to their observation regions in a plotted DT and further visualizations. The NN2EQCDT algorithm was then further evaluated by explaining a learned attacker policy model to show that one can ensure that a policy-based FF-DNN has not learned unknown unknown strategies.

Item Type:	Thesis (Masters)
Additional Information:	NN2EQCDT paper: https://www.thinkmind.org/index.php?view=article&articleid=cognitive_2023_1_160_40107 NN2EQCDT code repo: https://gitlab.com/arl-experiments/nn2eqcdt/-/tree/paper?ref_type=heads NN2EQCDT video presentation: https://www.youtube.com/watch?v=CEm9uWkfxiM
Uncontrolled Keywords:	Explainability of Deep Reinforcement Learning (XRL), Deep Reinforcement Learning (DRL), Deep Learning (DL), Deep Neural Network (DNN), Decision Tree (DT), Critical National Infrastructure (CNI), Power grid, Voltage control, Reactive Power Injection, Satisfiability Modulo Theories (SMT), Equivalent Transformation, ARL Framework, Trust, Resilience
Subjects:	Generalities, computers, information > Computer science, internet
Divisions:	School of Computing Science, Business Administration, Economics and Law > Department of Computing Science
Date Deposited:	23 Aug 2023 08:58
Last Modified:	23 Aug 2023 08:58
URI:	https://oops.uni-oldenburg.de/id/eprint/5840
URN:	urn:nbn:de:gbv:715-oops-59218
DOI:
Nutzungslizenz:

Actions (login required)

View Item