Article

Article title COMBINED COMPUTER VISION SYSTEM IN FIREFIGHTING ROBOT CONTROL SYSTEM
Authors A.V. Vazaev, V.P. Noskov, I.V. Rubtsov, S.G. Tsarichenko
Section SECTION V. VISION SYSTEMS
Month, Year 01-02, 2017 @en
Index UDC 007:621.865.8
DOI
Abstract Firefighting and reconnaissance robots further development connection with autonomy increasing is substantiated based on currently applied robots’ types, parameters and capabilities. In this paper using of extended environment model based on combined computer vision system data for firefighting robots motion planning and attached implements control is proposed. Extended model build and usage in navigation, motion planning and attached implements operation principles is shown. Human operator interface enhancements based on extended model is shown. Mathematical tool for firefighting hose stream detection and stream trajectory parameters calculation is provided. Created software efforts is provided in: image segmentation for heat source detection and firefighting hose stream trajectory parameters calculation, autonomous hose control signals calculating based on combined image from computer vision system. Full-scale experiment results in real working area model creation based on combined computer vision system data including mutually calibrated LiDAR sensor, color camera and thermovision camera images is provided. Such a sensor combination provides geometrical environment model with color and thermal information, which provides more accurate and simple solution of firefighting hose stream detection, fire detection. Possibility of firefighting robot working zone detecting based on scene geometry and hose stream trajectory is shown. Possible additional data visualization for operator knowledge during remote control is provided, including robot trajectory, possible working zone, fire position and hose stream trajectory. Full-scale experiment results given in this paper admit to make a conclusion that using of combined color-thermal-distance images allows to increase firefighting mobile robot autonomy by providing operator with additional information and using control system to solve some particular tasks.

Download PDF

Keywords Mobile robot; autonomous control system; combined computer vision system; environment model; recognition; classification.
References 1. Lakota N.A., Noskov V.P., Rubtsov I.V., Lundgren Ya.-O., Moor F. Opyt ispol'zovaniya elementov iskusstvennogo intellekta v sisteme upravleniya tsekhovogo transportnogo robota [Experience in the use of elements of artificial intelligence in the control system of shop trans-portation robot], Mekhatronika [Mechatronics], 2000, No. 4, pp. 44-47.
2. Kalyaev A.V., Noskov V.P., Chernukhin Yu.V., Kalyaev I.A. Odnorodnye upravlyayushchie struktury adaptivnykh robotov [Homogeneous control structures for adaptive robots]. Moscow: Nauka, 1990, 147 p.
3. Noskov V.P., Noskov A.V. Navigatsiya mobil'nykh robotov po dal'nometricheskim izobrazheniyam [Navigation of mobile robots in telematicheskikh images], Mekhatronika, avtomatizatsiya, upravlenie [Mechatronics, automation, control], 2005, No. 12, pp. 16-21.
4. Noskov V.P., Rubtsov I.V. Opyt resheniya zadachi avtonomnogo upravleniya dvizheniem mobil'nykh robotov [The experience of solving the problem of Autonomous motion control of mobile robots], Mekhatronika, avtomatizatsiya, upravlenie [Mechatronics, automation, control], 2005, No. 1, pp. 21-24.
5. Dementey V.P., Letkina N.Yu., Noskov V.P., Tsarichenko S.G. Intellektualizatsiya mo-bil'nykh pozharno-spasatel'nykh robotizirovannykh kompleksov [Intellectualization of mobile fire-rescue robotic systems], Trudy KhKhI Mezhdunarodnoy nauchno-tekhnicheskoy konferentsii «Ekstremal'naya robototekhnika» [Proceedings of XXI International scientific-technical con-ference "Extreme robotics"]. St. Petersburg: Izd-vo «Politekhnika-servis», 2010, pp. 40-46.
6. Kalinin A.V., Noskov V.P., Rubtsov I.V. Sistema avtovozhdeniya tanka T-72B [The system of driving tanks T-72B], Vestnik MGTU im. N.E. Baumana, seriya «Mashinostroenie». Spetsial'nyy vypusk «Spetsial'naya robototekhnika i mekhatronika» [Vestnik MGTU im. N.Uh. Bauman, series "Engineering". A special edition of "Special robotics and mechatronics"], 2012, pp. 134-148.
7. Noskov V.P., Rubtsov I.V. Klyuchevye voprosy sozdaniya intellektual'nykh mobil'nykh robotov [Key issues in the development of intelligent mobile robots], Vestnik MGTU im. N.E. Baumana. Seriya «Mashinostroenie» [Vestnik MGTU im. N.Uh. Bauman. Series "Engineering"], 2013.
8. Buyvolov G.A., Noskov V.P., Rurenko A.A., Raspopin A.N. Apparatno-algoritmicheskie sredstva formirovaniya modeli problemnoy sredy v usloviyakh peresechennoy mestnosti [Hardware and algorithmic means of forming a model of the problem environment in terms of terrain], Sb. nauchn. tr. Upravlenie dvizheniem i tekhnicheskoe zrenie avtonomnykh transportnykh robotov [Collection of scientific papers "motion Control and machine vision Autonomous robot vehicles"]. Moscow: IFTP. 1989, pp. 61-69.
9. Noskov A.V., Rubtsov I.V., Romanov A.Yu. Formirovanie ob"edinennoy modeli vneshney sredy na osnove informatsii videokamery i dal'nomera [The formation of the joint model of the external environment on the basis of information of the camera and the range finder] Mekhatronika, avtomatizatsiya, upravlenie [Mechatronics, automation, control], 2007, No. 8, pp. 2-5.
10. Kaz'min V.N., Noskov V.P. Vydelenie geometricheskikh i semanticheskikh ob"ektov v dal'nometricheskikh izobrazheniyakh dlya navigatsii robotov i rekonstruktsii vneshney sredy [The selection of geometric and semantic objects in telematicheskikh images for navigation of robots and reconstruction of the external environment], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2015, No. 10 (171), pp. 71-83.
11. Vazaev A.V., Noskov V.P., Rubtsov I.V., Tsarichenko S.G. Raspoznavanie ob"ektov i tipov opornoy poverkhnosti po dannym kompleksirovannoy STZ [Object detection and ground type classification with combined computer vision system], ] Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2016, No. 2 (175), pp. 127-139.
12. Gorban' Yu.I. Pozharnye roboty i stvol'naya tekhnika v pozharnoy avtomatike i pozharnoy okhrane [.Fire-fighting robots and conventional technique in fire automatics, and fire protection]. Moscow: Pozhnauka, 2013, 352 p.
13. Wang W., Pottmann H., Liu Y. Fitting B-spline Curves to Point Clouds by Curvature-based Squared Distance Minimization, ACM Trans. Graph., 2006, Vol. 25, No. 2, pp. 214-238.
14. Dimitrov A., Gu R., Golparvar-Fard M. Non-Uniform B-Spline Surface Fitting from Unor-dered 3D Point Clouds for As-Built Modeling, Computer-Aided Civil and Infrastructure Engi-neering, 2016, Vol. 31, No. 7, pp. 483-498.
15. Maekawa T., Ko K.H. Surface Construction by Fitting Unorganized Curves, Graph. Models., 2002, Vol. 64, No. 5, pp. 316-332.
16. Pottmann H., Leopoldseder S., Hofer M. Approximation with Active B-Spline Curves and Surfaces, Proceedings of the 10th Pacific Conference on Computer Graphics and Applications. Washington, DC, USA: IEEE Computer Society, 2002, pp. 8.
17. Al-Moadhen A. et al. Improving the Efficiency of Robot Task Planning by Automatically Inte-grating Its Planner and Common-Sense Knowledge Base, Knowledge-Based Information Systems in Practice / ed. Tweedale J.W. et al. Springer International Publishing, 2015, pp. 185-199.
18. Ji J., Chen X. From structured task instructions to robot task plans, IC3K 2013; KEOD 2013 - 5th International Conference on Knowledge Engineering and Ontology Development, Pro-ceedings, 2013, pp. 237-244.
19. Galindo C., Fernandez-Madrigal J.-A., Gonzalez J. Improving efficiency in mobile robot task planning through world abstraction, IEEE Transactions on Robotics, 2004, Vol. 20, No. 4,
pp. 677-690.
20. Milella A. et al. Combining radar and vision for self-supervised ground segmentation in outdoor environments, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011, pp. 255-260.

Comments are closed.