|Article title||ORGANIZATION OF COMPUTATIONS IN HETEROGENOUS DISTRIBUTED ENVIRONMENTS|
|Section||SECTION IV. CLOUD COMPUTING|
|Month, Year||12, 2016 @en|
|Abstract||Modern scientific research is inextricably linked to the carrying out of complex calculations and the use of high performance computing resources. Currently available to researchers, there are a variety of such computing resources, including servers and the individual workstations, clusters and supercomputers, grid systems and cloud infrastructures. Each of these types of resources has its own characteristics, advantages and disadvantages. Sharing computing capacity of these resources within the heterogeneous distributed computing environments (HDCE) would signifi-cantly increase research productivity and resource efficiency. This article is an attempt to describe in the most general formulation the problems of organization of computations in HDCE, including the main types of resources and their characteristics, types of environments and their use, classes of applications and user requirements. HDCE refers to arbitrary combinations of distributed com-puting resources of various types. These resources can significantly differ from each other in their characteristics, access mechanisms, administrative affiliation, groups of served users, availability periods, costs, policies and quality of service levels. HDCE can have temporary (for solution of a certain problem) or permanent nature, and vary according to the number of users (personal or multiuser environments) and the range of resource providers. Over the past decade, the range of HDCE is significantly expanded while their structure has become more complex. There are new types of resources being HDCE themselves and allowing building new environments at a higher level of abstraction. At the same time there are new requirements to the methods and tools used for management of computations in HDCE, such as optimization of expenses for renting cloud computing resources while meeting the specified deadline to carry out the computations. The high degree of heterogeneity and the dynamic nature of HDCE, the proliferation of cloud services, a variety of usage scenarios and types of applications require the improvement of existing and development of new approaches to solving problems related to the organization of computations in such environments.|
|Keywords||Distributed computing; heterogeneous computing environments; integration of computing resources; grid; cloud computing.|
|References||1. Catlett C.E. Metacomputing, Communications of the ACM, 1992, Vol. 35, No. 6, pp. 44-53.
2. Kovalenko V., Koryagin D. Vychislitel'naya infrastruktura budushchego [Future computing infrsastructure], Otkrytye sistemy [Open systems], 1999, No. 11-12, pp. 45-52.
3. Voevodin V.V., Zholudev Yu.A., Sobolev S.I., Stefanov K.S. Evolyutsiya sistemy metakomp'yutinga X-Com [Evolution of X-Com metacomputing system], Vestnik Nizhegorodskogo universiteta im. N.I. Lobachevskogo [Vestnik of Lobachevsky University of Nizhni Novgorod], 2009, No. 4.
4. Anderson D.P., Fedak G. The computational and storage potential of volunteer computing, Cluster Computing and the Grid, 2006. CCGRID 06. Sixth IEEE International Symposium on. IEEE, 2006, Vol. 1, pp. 73-80.
5. Lovas R., Afanas'ev A.P., Voloshinov V.V., Posypkin M.A., Sukhoroslov O.V., Khrapov N.P. Uvelichenie vychislitel'noy moshchnosti raspredelennykh sistem s pomoshch'yu grid-sistem iz personal'nykh komp'yuterov [Increasing the computing performance of distributed systems with the help of desktop grids], Parallel'nye vychislitel'nye tekhnologii (PaVT’2011): Trudy Mezhdunarodnoy nauchnoy konferentsii [Parallel computational technologies
(PaVT’2011): Proceedings of the International scientific confer]. Chelyabinsk: YuUrGU, 2011, pp. 6-14.
6. Foster I., Kesselman C. (ed.). The Grid 2: Blueprint for a new computing infrastructure. Else-vier, 2003.
7. Afanas'ev A.P., Voloshinov V.V., Rogov S.V., Sukhoroslov O.V. Razvitie kontseptsii raspredelennykh vychislitel'nykh sred [The development of distributed computing environment concept], Problemy vychisleniy v raspredelennoy srede: organizatsiya vychisleniy v global'nykh setyakh: Trudy ISA RAN [Problems computing in a distributed environment: the organization of computing in global networks: Proceedings of ISA RAS]. Moscow: ROKhOS, 2004, pp. 6-105.
8. Kovalenko V.N., Koryagin D.A. Grid: istoki, printsipy i perspektivy razvitiya [Grid: origins, principles and prospects], Informatsionnye tekhnologii i vychislitel'nye sistemy [Information technology and computer systems], 2008, No. 4, pp. 38-50.
9. Emel'yanov S.V., Afanas'ev A.P., Voloshinov V.V., Grinberg Ya.R., Krivtsov V.E., Sukhoroslov O.V. Realizatsiya Grid-vychisleniy v srede IARnet [Implementation of grid computing in IARnet environment], Informatsionnye tekhnologii i vychislitel'nye sistemy [Information technology and computer systems], 2005, No. 2, pp. 61-75.
10. Radchenko G.I. Tekhnologiya postroeniya problemno-orientirovannykh ierarkhicheskikh obolochek nad inzhenernymi paketami v grid-sredakh [Technology for building problem-oriented hierarchical wrappers of engineering packages in grid environments], Sistemy upravleniya i informatsionnye tekhnologii [Control systems and information technology], 2008, No. 4, pp. 57-61.
11. Bukhanovskiy A.V., Koval'chuk S.V., Mar'in S.V. Intellektual'nye vysokoproizvodi-tel'nye programmnye kompleksy modelirovaniya slozhnykh sistem: kontseptsiya, arkhitektura i primery realizatsii [Intellectual high-performance software packages for modeling complex systems: concept, architecture and examples of implementation], Izvestiya vuzov. Priborostroenie [Journal of Instrument Engineering], 2009, Vol. 52, No. 10, pp. 5-24.
12. Armbrust M. et al. A view of cloud computing, Communications of the ACM, 2010, Vol. 53, No. 4, pp. 50-58.
13. Buyya R. et al. Cloud computing and emerging IT platforms: Vision, hype, and reality for de-livering computing as the 5th utility, Future Generation computer systems, 2009, Vol. 25,
No. 6, pp. 599-616.
14. Nair S. K. et al. Towards secure cloud bursting, brokerage and aggregation, Web services (ecows), 2010 ieee 8th european conference on. IEEE, 2010, pp. 189-196.
15. Astaf'ev A.S., Afanas'ev A.P., Lazarev I.V., Sukhoroslov O.V., Tarasov A.S. Nauchnaya servis-orientirovannaya sreda na osnove tekhnologiy Web i raspredelennykh vychisleniy [Academic service-oriented environment based on web technologies and distributed computing], Nauchnyy servis v seti Internet: masshtabiruemost', parallel'nost', effektivnost': Trudy Vserossiyskoy superkomp'yuternoy konferentsii [Scientific service in the Internet: scalability, parallelism, efficiency: Proceedings of all-Russian supercomputer conference]. Moscow: MGU, 2009, pp. 463-467.
16. Bukhanovskiy A.V., Vasil'ev V.N., Vinogradov V.N., Smirnov D.Yu., Sukhorukov S.A., Yapparov T.G. CLAVIRE: perspektivnaya tekhnologiya oblachnykh vychisleniy vtorogo pokoleniya [CLAVIRE: perspective second generation cloud computing technology], Izvestiya vysshikh uchebnykh zavedeniy. Priborostroenie [Journal of Instrument Engineering], 2011, Vol. 54, No. 10.
17. Samovarov O.I., Gaysaryan S.S. Arkhitektura i osobennosti realizatsii platformy UniHUB v modeli oblachnykh vychisleniy na baze otkrytogo paketa OpenStack [Architecture and details of implementation of UniHUB platform in cloud computing model on the base of OpenStack package], Trudy Instituta sistemnogo programmirovaniya RAN [Proceedings of Institute for system programming of Russian Academy of Sciences], 2014, Vol. 26, No. 1.
18. Sukhoroslov O.V. Integratsiya vychislitel'nykh prilozheniy i raspredelennykh resursov na baze oblachnoy programmnoy platformy [Integration of computing applications and distributed re-sources on the base of cloud software platform], Programmnye sistemy: teoriya i prilozheniya [Program systems: theory and applications], 2014, Vol. 5, No. 4 (22), pp. 171-182.
19. Polyakov S.V., Vyrodov A.V., Puryr'kov D.V., Yakobovskiy M.V. Oblachnyy servis dlya resheniya mnogomasshtabnykh zadach nanotekhnologii na superkomp'yuternykh sistemakh [Cloud service for solution of multi-scale nanotechnology problems on supercomputer systems], Trudy Instituta sistemnogo programmirovaniya RAN [Proceedings of Institute for system programming of Russian Academy of Sciences], 2015, Vol. 27, No. 6.
20. Velikhov V.E., Klimentov A.A., Mashinistov R.Yu., Poyda A.A., Ryabinkin E.A. Integratsiya geterogennykh vychislitel'nykh moshchnostey NITs «Kurchatovskiy institut» dlya provedeniya masshtabnykh nauchnykh vychisleniy [Integration of heterogeneous computing resources at nrc «Kurchatov institute» for large-scale scientific computations], Izvestiya YuFU. Tekhnicheskie nauki [Izvestiya SFedU. Engineering Sciences], 2016, No. 11 (184), pp. 88-99.
21. Foster I. Globus toolkit version 4: Software for service-oriented systems, IFIP international conference on network and parallel computing. Springer Berlin Heidelberg, 2005, pp. 2-13.
22. Laure E. et al. Programming the Grid with gLite, Computational methods in science and tech-nology, 2006, Vol. 12, No. 1, pp. 33-45.
23. Demichev A.P., Kryukov A.P., Shamardin L.V. Printsipy postroeniya grid s ispol'zova-niem RESTful-veb-servisov [Principles of building grid using RESTful web services], Programmnye produkty i sistemy [Software products and systems], 2009, No. 4.
24. Kondo D. et al. Characterizing and evaluating desktop grids: An empirical study, Parallel and Distributed Processing Symposium, 2004. Proceedings. 18th International. IEEE, 2004, pp. 26.
25. Ivashko E.E. Desktop Grid korporativnogo urovnya [Enterprise desktop grids], Programmnye sistemy: teoriya i prilozheniya [Program systems: theory and applications], 2014, Vol. 5, No. 1,
26. Anderson D.P. et al. SETI@home: an experiment in public-resource computing, Communica-tions of the ACM, 2002, Vol. 45, No. 11, pp. 56-61.
27. Zaikin O.S., Posypkin M.A., Semenov A.A., Khrapov N.P. Opyt organizatsii dobrovol'nykh vychisleniy na primere proektov OPTIMA@home i SAT@home [Experience in volunteer computing on the example of OPTIMA@home and SAT@home projects], Vestnik Nizhegorodskogo universiteta im. N.I. Lobachevskogo [Vestnik of Lobachevsky University of Nizhni Novgorod], 2012, No. 5-2.
28. Andrade N. et al. OurGrid: An approach to easily assemble grids with equitable resource shar-ing, Workshop on Job Scheduling Strategies for Parallel Processing. Springer Berlin Heidel-berg, 2003, pp. 61-86.
29. Kurze T. et al. Cloud federation, CLOUD COMPUTING, 2011, Vol. 2011, pp. 32-38.
30. Cirne W. et al. Grid computing for bag of tasks applications, In Proc. of the 3rd IFIP Confer-ence on E-Commerce, E-Business and EGovernment, 2003.
31. Taylor I.J. et al. Workflows for e-Science: scientific workflows for grids. Springer Publishing Company, Incorporated, 2014.
32. Lazarev I.V., Sukhoroslov O.V. Ispol'zovanie workflow-metodologii dlya opisaniya protsessa raspredelennykh vychisleniy [Using workflow methodology for description of the process of distributed computations], Problemy vychisleniy v raspredelennoy srede: Modeli obrabotki i predstavleniya dannykh. Dinamicheskie sistemy: Trudy ISA RAN [Computing problems in a distributed environment: models of processing and presentation of data. Dynamical systems: Proceedings of ISA RAS], 2005, Vol. 14, pp. 254-255.
33. Sukhoroslov O.V. Novye tekhnologii raspredelennogo khraneniya i obrabotki bol'shikh massivov dannykh [New technologies for distributed storage and processing of big datasets]. Moscow: Institut sistemnogo analiza RAN, 2008, 40 p.
34. Chen C.L.P., Zhang C.Y. Data-intensive applications, challenges, techniques and technologies: A survey on Big Data, Information Sciences, 2014, Vol. 275, pp. 314-347.
35. Beckman P. et al. SPRUCE: A system for supporting urgent high-performance computing, Grid-Based Problem Solving Environments. Springer US, 2007, pp. 295-311.
36. Bukhanovskiy A.V., Zhitnikov A.N., Petrosyan S.G. Vysokoproizvoditel'nye tekhnologii ekstrennykh vychisleniy dlya predotvrashcheniya ugrozy navodneniy [High-performance tech-nologies of urgent computing for preventing the threat of flooding], Izvestiya vysshikh uchebnykh zavedeniy. Priborostroenie [Journal of Instrument Engineering], 2011, Vol. 54, No. 10.