This is followed by the PCA analysis results, we report the pivot and the 2 principle components. This follows two counterexample sets, one randomly generated from the error table and the other with k closest samples. You should see a table labeled Error Table a collection of all falsified samples in the first terminal where you ran python falsifier.py. Its True if the NN correctly classifies the image. Here rho represents the qualitative (boolean) satisfaction. Use the PCA analysis on the samples to generate new samplesĭuring the running of the falsifier you should the samples and the associated value of the specification satisfaction (rho). Top k closest (similar) sampler from the error_table Randomly sample samples from the error_table We have introduced three techniques to generate new images for the NN re-training: We can further analyse the error_table to generate images for retraining the NN. The falsifying samples are stored in the data structure error_table. At the end of the runs, you should see “End of all classifier calls” in the terminal where you ran python classifier.py. The falsifier runs for 20 iterations, you can change this by modifying MAX_ITERS in examples/data_augmenatation/falsifier.py. Then in first one run python falsifier.py and wait till you see “Initialized sampler” in the terminal then run python classifier.py in other one. Running the falsifier: Open two terminal shells and go to cd data_augmentation in each of them. Sample space: Image background (37 backgrounds), number of cars- (x, y) position and type of car, overall image brightness, color, contrast, and sharpness.Įxamples/data_augmenatation/falsifier.py : Defines the sample space and type of falsifier (sampler and number of iterations)Įxamples/data_augmentation/classifier.py : Interface to the picture renderer and instantiate the NN Task: Falsify the NN trained on the synthetic images generated by the picture rendered We implemented our own picture renderer which generates images by sampling from a low dimensional modification (sample) space. We re-create the data augmentation example from this paper. In this example we try to falsify a Neural Network (NN) trained to detect images of cars on roads.
You should see two tables in the first terminal where you ran python examples/lanekeeping_LQR/lanekeeping_falsifier.py, labeled Falsified Samples a collection of all falsified samples and Safe Samples a collection of all the samples which were safe. Rho represents the quantitative satisfaction of the specification such that the sample satisfies the specification if the rho is positive or falsifies the specification if the rho is negative. At the end of the runs, you should see “End of all simulations” in the terminal where you ran python examples/lanekeeping_LQR/lanekeeping_simulation.py.ĭuring the running of the falsifier you should the samples and the associated value of the specification satisfaction (rho). The falsifier runs for 20 iterations, you can change this by modifying MAX_ITERS in examples/lanekeeping_LQR/lanekeeping_falsifier.py. Then in first one run python examples/lanekeeping_LQR/lanekeeping_falsifier.py and wait till you see “Initialized sampler” in the terminal then run python examples/lanekeeping_LQR/lanekeeping_simulation.py in the other one. Running the falsifier: To run this example open two terminal shells and go to cd verifai/simulators/car_simulator in each of them. Verifai/simulators/car_simulator/examples/lanekeeping_LQR/lanekeeping_simulation.py : Defines the controller and the simulation environment
Verifai/simulators/car_simulator/examples/lanekeeping_LQR/lanekeeping_falsifier.py : Defines the sample space and type of falsifier (sampler and number of iterations) Sample space: Initial x-position, angle of rotation, and cruising speed. Task: Falsify the LQR lane keeping controller In this example we have a car (in red) whose task is to stay within its lane using an LQR controller.
WEBOTS TUTORIAL PDF SIMULATOR
VerifAI comes with an inbuilt simulator developed from this car simulator.
0 kommentar(er)
