The Importance of Virtual Reality in the Conservation of Cultural Heritage: An Examination of the Penataran Temple Case Study in Blitar Regency, East Java

The visualization of digital archives of plays plays an increasingly important role in the preservation and analysis of cultural heritage. Alongside these visualization digital archives, numerous meaningful applications can be implemented, such as appearance guidance, computer-aided design, geographic information systems, and virtual reality applications. The effectiveness of these applications depends on the efficient utilization of digitalization techniques. Utilizing technologies like 3D scanning and modeling facilitates the efficient acquisition and preservation of digital data related to the existing cultural heritage assets. When dealing with cultural heritage that has been lost over time, a picture-based approach can be used to reconstruct 3D models. However, in many cases, only monocular photos of certain objects are available. Therefore, a customized reconstruction method with single-picture scenarios becomes crucial. To address this, a new approach of visualizing in three dimensions (3D) has been introduced to proficiently reconstruct and visualize cultural heritage from solitary monocular photos. The efficacy of this recommended method has been demonstrated through its successful implementation at the Penataran Temple in the Regency of Blitar, East Java, yielding promising results. Point cloud in three dimensions (3D) emerges as a favored representation for digital archives documenting cultural heritage sites and artifacts. This point cloud essentially consists of a collection of points defined by coordinate values and color attributes, acquired through laser scanning or photogrammetry techniques. Penataran Temple, situated in the Regency of Blitar, East Java, represents the largest complex temple in the province of East Java, Indonesia. Positioned on the slopes of Mount Kelud in Penataran Village, District Nglegok, Regency Blitar, this temple holds significant religious importance for the Hindu community and serves as a potential tourist destination that contributes to the economic development of the local community. The enhancement of the temple includes various tourist attractions such as Sumberjati Temple, Kali Cilik Temple, Sumbernanas Temple, Mleri Temple, Bacem I and II Temples, Rambut Monte Temple, Plumbangan Temple, Tepas Temple, Sirah Kencong Temple, Watu Temple Tumbuk, and Gambar Wetan Temple (Nugroho, 2002) Penataran Temple serves as a place of religious significance, where individuals come to engage in acts of worship and prayer. The Hayuning Wedar Ceremony, an annual event conducted at Penataran Temple, is designed with the purpose of offering prayers for the collective well-being of the community, aiming to achieve salvation, tranquility, and abundance. This ceremonial event fulfills various roles, such as religious observance, socio-economic engagement, and the preservation of cultural practices (Rahayu et al., 2018). To enhance the potential of Penataran Temple as a tourist destination, an initiative was undertaken to develop park tours around the temple. This endeavor aims to harness the inherent natural resources and beauty of the area, while also reinforcing the values of national identity and cultural heritage (Nugroho, 2002). Research is furthermore conducted to identify and analyze the visual state of the Penataran Temple surroundings, indicating that the visual aspects of the temple grounds are aesthetically pleasing and possess potential for advancement as a tourist destination (Baptisa et al., 2019). Serving as a wellspring of knowledge about life, the Penataran Temple also functions as a site for the documentation and categorization of the diverse range of plant and animal species depicted in the temple's reliefs (Widiastuti et al., 2012). The development of the branding of Penataran Temple was also undertaken to enhance public awareness of the temple's history, enabling visitors to appreciate the beauty of the temple while gaining an understanding of its historical significance (Fanani et al., 2016). Figure 1. Penataran Temple, located in the Regency of Blitar in East Java, is depicted in the photograph. The Penataran Temple serves as a valuable repository of knowledge as it contains a comprehensive, precise, and reliable record of the various forms of plant and animal life depicted in the temple's reliefs (Widiastuti et al., 2012). A study was conducted t identify and analyze the environmental visual conditions of a temple, revealing the potential for developing the temple area as a tourist destination (Fanani et al., 2016). The Penataran Temple serves as a site for the documentation and inventory of diverse aspects of life, as depicted in the temple reliefs (Widiastuti et al., 2012). The tourism sector is considered to be a pivotal industry for economic growth, particularly in terms of generating foreign exchange. The advancement of tourism is intricately linked with the conservation of national values, cultural development, and the utilization of the full range of natural resources available within local areas (Nugroho, 2002). Examine the objective of enhancing the significance of Penataran Temple, not solely as a place of worship, but also as a repository of wisdom encompassing various aspects of life such as nature, environment, society, and culture (Widiastuti et al., 2012). As a source of knowledge on life, Penataran Temple has the potential to serve as a location for the documentation and inventory of the diverse biodiversity, encompassing flora and fauna, depicted in the temple reliefs (Widiastuti et al., 2012). Research is also conducted to identify and analyze the visual condition of the Penataran Temple environment. This research reveals that the temple area possesses potential for development as a tourist destination (Fanani et al., 2016). The development of branding for Penataran Temple was also implemented to enhance public awareness of the temple's history, allowing visitors to appreciate the beauty of the temple while gaining insights into its historical significance (Fanani et al., 2016). Penataran Temple, also referred to as Palah Temple, represents a grand Hindu temple situated in East Java, Indonesia. This particular temple holds the distinction of being the largest complex in the area, situated on the southwestern slopes of Mount Kelud, adjacent to the north of Blitar, at an elevation of 450 meters above sea level. Constructed during the rule of King Srengga from the Kediri Kingdom around 1200 AD, the temple remained in use until the era of King Wikramawardhana from the Majapahit Kingdom around 1415. Penataran Temple is renowned for its distinct linear layout, deviating from the concentric mandala designs prevalent in Central Javanese temples like Sewu Temple. This site has been nominated as a UNESCO World Heritage Site since October 19, 1995. The intricate temple complex comprises a series of parallel structures along the northwest-southeast axis, with a river originating from Mount Kelud flowing behind the main temple towards the east (Pratiwi, 2016; Suhadak et al., 2022) A technique for 3D reconstruction with a specific emphasis on immersive visualization was employed to unveil obscured portions of the Penataran Temple. This process involved transforming segments that were not initially visible into cloud data, which originated from historical monocular photographs taken by Kassian Cephas in 1890 (Pan et al., 2020). In this research, we hereby introduce a system utilizing virtual reality (VR) for the digitization of Penataran Temple as a cultural archive. VR proves to be suitable for the observation and analysis of preserved cultural sites and artifacts. Within this section, we delineate various instances of utilizing VR for the digital conservation of cultural heritage properties

Research methodologies that are applicable for conducting investigations are diverse. This encompasses a wide range of techniques and approaches: (1) Photogrammetry involves the integration of a camera within an Unmanned Aerial Vehicle (UAV) to capture images and generate a 3D point cloud representing the entire Penataran Temple. The methodology of photogrammetry is employed to measure and reconstruct the physical dimensions of objects through the geometric analysis of the captured images. (2) The precision of measurements in photogrammetry requires more complex procedures, especially in certain parts of the Penataran Temple structure. This approach enables a more suitable 3D mapping of the specified area within the temple. (3) A deep learning technology is utilized to process and synthesize the derived 3D data of the concealed relief panels illustrated in the 2D monocular photos archived. Deep learning strategies are implemented to identify and reconstruct hidden reliefs in three dimensions. Data collected from three sources is aggregated to generate a 3D Point Cloud, serving as a digital archive for Penataran Temple. A virtual reality (VR) system was developed to allow individuals to observe the temple from two distinct viewpoints: a first-person perspective and an aerial view angle. By utilizing an intuitive interface for VR users, it is easy for users to switch between these perspectives. This system is constructed using a 3D simulation engine (3D game engine). 3.1 Variability of Data Sets Within the Digital Archives of Penataran Temple is A Topic of Interest Currently, the researcher is engaged in the design and development of a digital repository for Penataran Temple. Each section of the temple will be subject to measurement and documentation utilizing suitable techniques specific to the area. Subsequently, the archived data sets exhibit variability and a range of configurations. Nevertheless, the transformation of each data collection into cloud data points allows for the integration of a comprehensive data set based on key points. 1) 3D Measurement with the SFM-MVS Method for Point Cloud Data Temple 3D measurements have become increasingly popular as a method for acquiring point cloud data from architectural structures. The main approach for conducting 3D building measurements involves laser-based techniques and photogrammetry methodology. Within the scope of this investigation, the Structure from Motion-Multi View Stereo (SFM-MVS) technique, which falls under the umbrella of photogrammetry, is utilized to generate point cloud data from various parts of a temple. The SFM-MVS method entails the use of information captured through photogrammetry with a digital camera, which is subsequently processed in Agisoft Metashape, a specialized software tool in photogrammetry and multiview stereo, to merge multiple photogrammetric datasets and produce a 3D model. Unmanned Aerial Vehicles (UAVs) provide a viable means for acquiring photogrammetric data from hard-to-reach areas, such as rooftops. By integrating data obtained by UAVs, a comprehensive imaging of the structure can be achieved. Sixty images of the temple were captured entirely using an Unmanned Aerial Vehicle (UAV) and a digital camera (DJI FC3008S), providing a resolution of 4,000 x 3,000 pixels. The 3D model of the entire temple was created using photogrammetry techniques. During the process of collecting photogrammetric data, it is assumed that the data will be converted into a 3D format. The careful merging of overlapping images is crucial for the photogrammetric measurement procedure. Therefore, meticulous attention must be paid to capturing images sequentially to facilitate the precise generation of 3D data. The equipment necessary for carrying out photogrammetric measurements primarily includes a digital camera. In comparison to laser-based measurements, this method eliminates the need for complex preparations and offers cost-effective advantages. Figure 3. Example Camera Position For Reconstructing the Entire Temple Building Using Photogrammetry. Figure 4. Mysterious World Nations at Penataran Temple (chosen) 2) Creating a Point Cloud Temple using Digital Photogrammetry. Point clouds depicting intricate details of certain parts of a temple are generated through the application of photogrammetry, utilizing a large number of highresolution digital photos. The utilization of digital photogrammetry allows for the capture of precise details within a specified area of interest at close distances. In our research, we employ the RICOH GR III camera, which offers a resolution of 6,000 x 4,000 pixels, for the purpose of acquiring objective pictures. A monopod is used to facilitate the stabilization of the camera during the process of data collection (see Figure 3). Figure 4 illustrates the optimal position of the camera for reconstructing a detailed point cloud within a specified area of interest 3) Reconstruction of three-dimensional hidden relief using a single camera. Rebuilding 3D topography from a single image of the Penataran Temple necessitates the utilization of specialized tools and methodologies in the fields of photogrammetry and computer vision. Various technologies are available for this purpose, with specific applications tailored towards achieving defined objectives: (1) Structure from Motion (SfM): The methodology known as Structure from Motion (SfM) is a photogrammetric technique utilized for the reconstruction of a threedimensional structure from a two-dimensional image. This approach involves the identification of common features across multiple images to estimate the position and orientation of the camera. SfM is applicable for generating a 3D model of objects and scenes using either a single camera or multiple cameras. (2) Multi-View Stereo (MVS): Multi-View Stereo (MVS) is a technique employed to reconstruct the three-dimensional structure from a series of images captured from various perspectives. This method focuses on identifying suitable points within the images to determine the 3D geometry of the views. MVS is capable of producing detailed 3D models of objects and scenes with high accuracy. (3) Laser Scanning: Laser scanning is a method that utilizes laser scanners to capture the three-dimensional geometry of objects or environments. This technique operates by emitting laser light towards objects and detecting the reflections using sensors. The distance between the scanner and the object is calculated based on the time taken for the laser beam to return to the sensor. Laser scanning enables the acquisition of highly precise 3D models of objects and complex geometries within scenes. Pan et al. (2020) propose a method based on deep learning for reconstructing a 3D point cloud of hidden relief using monocular colored gray photos. The concept of 3D reconstruction relies on neural networks to estimate depth. Initially, the network is trained with a set of monocular photos to map the corresponding depths. The training data is derived from photogrammetry of visible relief point clouds. Subsequently, utilizing the trained neural network, one can estimate and map the appropriate depths for monocular photos lacking known depth information. The 3D point cloud is then created from the estimated map and depths imported into our VR system. As demonstrated by Eigen et al. (2014), a significant error arises in part due to the utilization of metric-based elemental moments. This phenomenon can be elucidated by the degree to which accurate predictions align with the average depth. The majority of the overall error stems from the estimation of the average scale within a given scene. Thus, in the context of deep learning, it is crucial to assess the depth estimation by examining the interconnections between various points in the scene, while disregarding the absolute global scale. In this study, we incorporate error invariant scales into the loss function, following the methodology established by Eigen. Specifically, for the mapping of predicted depth (y) and ground truth depth (y*), each comprising n pixels denoted by an index, the error-invariant mean square scale is defined as follows: 1 n D(y,y) = ZZ(logyi —logy; +a(¥,y"))? i=1 1 ay) =4 Z(logyf —logy:) i [€)) &) Inspired by mistakes scale scene This is for map predicted depth y and truth earth v*, loss per- sample training arranged as following: Ly = %Z @ - %(Z di>2 i d; =logy; — logy; Information: n is i 3 @) amount square difference between base 10 Z(IOE 10(y;) — log10(y;))* i=1 n n i=1 logarithm of every yi and yi *. For measure how much big difference between base 10 logarithm of every element yi and yi *. is difference between amount base 10 logarithm of all Z log 10(y;) — Z log10(y;) yi and amount base 10 logarithm of all yi *. For i=1 measure difference between the total logarithms of base 10 of all over element yi and all over element yi *. ni is multiplied constant with results The calculation above is also possible works as factor adjustment or weighting 3.2 VR Development Encompasses the Utilization of 3D Simulation Techniques and the Process Of Mapping Specific Points The study conducted utilized 3D simulation in machines to aid in the development of crucial VR functions. Modern machine simulation now possesses the capability to assist in dot mapping on the side of conventional polygon mapping, as well as a combination of the two types. The inherent challenges in utilizing point mapping through machine 3D simulation are essential in reducing data size by thinning the original points, a critical step in achieving real-time mapping. Furthermore, the subtraction of specific points is crucial to ensure that the quality of mapping remains at an optimal level. Next, every 3D point is grouped and projected onto the field image to create an intermediary image. Throughout the projection process, the aggregation of points per pixel is taken into account. A set of L intermediary images is generated using this method. Subsequently, the intermediary images of L are merged through averaging to create a final transparent and conclusive image. The number of points denoted as n, the surface area S of the wide segments, the dimensions of the point s, and the turbidity of the surface in each segment are given specific local markings: a:l—(l—%) Information: n 5) a = represents a variable or parameter sand S = size or scale n = the possibility of representing a number In method this, L available as a quality parameter picture Because reflect amount picture averaged intermediate. With arrange amount point local, n, us can control turbidity surface local a is appropriate with Equation 3. 1) The Function of Machine Simulation in the Development of Virtual Reality Incorporating Intriguing Point Clouds Attention Machine simulation is the process of developing integrated design environments for computer simulation. Initially, machine simulations were designed for computer graphics (CG) using polygons. However, the utilization of point clouds as input data has emerged, enabling the use of 3D point clouds. Machine simulations incorporate state-of-the-art computer graphics technology and advanced tools for rendering, animation, and crash detection. Additionally, they encompass a wide range of functions for constructing VR spaces and user interface capabilities to manage VR environments. Consequently, beyond the initial aim of production simulation, machines have found extensive applications in video production, scientific and technological visualization, as well as VR advancements in fields such as tourism, architecture, and medicine. Our aim is to implement VR environments created through machine simulation in a digital repository of cultural assets. Currently, various machine simulations are available, and we have utilized Unity to import, merge, and visualize point clouds, as it holds the largest market share. Unity also serves as a leading VR development platform, facilitating the creation of seamlessly compatible VR content for a variety of VR devices The three categories of point clouds outlined in Section 2 have been amalgamated and incorporated into the framework of our Virtual Reality (VR) research. Upon loading point cloud data, it is necessary to convert the point cloud into PLY format, and then integrate it into Unity by utilizing established assets such as Pex (Pex Importer/Renderer point cloud for Unity, nd). A decrease in frame rate within a VR environment may result in a phenomenon known as a "drunk journey." Nevertheless, the amalgamated point cloud derived from the original dataset encompasses over 100 million points. Displaying a point cloud of such magnitude in VR settings may have a unique impact on frame rate reduction. Consequently, a reduction in sample size was carried out on the three initial point cloud instances utilizing a data depletion technique based on Poisson Disk Sampling (PDS). PDS is an algorithm designed to generate a uniform point cloud by contracting the point cloud with a certain distance between points. In this study, the number of points in each point cloud was reduced to 4 million using PDS. This data reduction led to an enhancement in rendering speed to 30 frames per second (fps) in our VR system. Furthermore, the focus on point size was broadened to prevent a decrease in visibility that may occur after the sample subtraction. 2) The utilization of First Person View and Bird's Point of View in Virtual Reality Space is being examined. Two main categories of perspectives can be delineated within the realm of virtual reality (VR), particularly the bird's-eye view and the first-person perspective. The bird's-eye perspective allows for a comprehensive observation of the surroundings from various angles by the users. By manipulating the mouse, individuals are able to navigate and rotate the temple structure while adjusting the viewpoint to align with their preferences. Conversely, it is the individual who opts for the first-person perspective to make use of VR functionalities in order to explore and appreciate cultural landmarks with a heightened sense of immersion. A feature has been developed to ensure a seamless transition between these two perspectives through direct manipulation of the mouse. (1) Primary observer within a Virtual Reality (VR) framework. The observer initiates the facilitation of an immersive virtual tour of Penataran Temple by manipulating the mouse and keyboard. Through a subjective perspective, our virtual reality system enables individuals worldwide to virtually explore the Penataran World Heritage Site, without being limited by temporal or spatial constraints. To achieve this objective, it is crucial to ensure the interaction between the camera and the ground or the temple structure. The integration of a mechanism to detect collisions between the camera and the point cloud is accomplished by utilizing a network of translucent polygon representatives of the ground and stairs of the temple. This integration into the point cloud allows for smooth navigation and a wellorganized virtual reality experience (2) Viewpoint ascertained from an elevated vantage point. Perspective air offers a means to observe a temple from a vantage point covering all its structures, as depicted in sample pictures viewed by observers from specific angles. This vantage point provides a comprehensive outlook on the temple and its surrounding areas from various perspectives. Such views are invaluable for gaining a deeper understanding of the cultural heritage, especially for scientific and pedagogical purposes, particularly during school visits to Penataran Temple, a popular destination for primary and secondary school students in Indonesia. It serves as a preparatory exercise before the visit and a reflective tool afterwards. Our collaboration with local community centers and Indonesian archaeologists has facilitated the accessibility of our virtual reality system to a wider audience, promoting sustainable educational opportunities for all individuals. (3) Penataran Temple's 3D relief sculpture depicts intricate carvings and details. Relief is discovered at Penataran Temple, which holds significant cultural interest. It provides a description of the habits and ways of life in Indonesia for the development of the temple. In addition to the concealed reliefs on the surface of the land, which have been transformed into representations in three dimensions through sophisticated machine learning techniques, our efforts also encompass the assessment and visualization of relief panels that are visible at elevated levels. These findings will be integrated into our virtual reality (VR) system. The triple relief panel illustration, depicting the reconstructed dimensions, is presented in Figure 4.

The experimental results will be determined based on estimated depth, reconstruction, and transparent visualization. A comparison of experiments will be conducted between two distinct datasets. One dataset, consisting of a smaller amount of data, will be referred to as Test1, while the other dataset, containing a larger amount of data, will be referred to as Test2. Table 1. Comparison of Estimation Results Depth with Test 1 and Test 2 Estimated Results Depth Trial /Trial i Higher Reconstruction Lower Results @ | « | a3 | RMSE | RMSELOG | “verage distance (meters) Test1 Test2 Information: 0.2211 | 0.3803 | 0.4899 | 9.9221 | 0.3711 0.2313 | 0.3846 | 0.5988 | 9.9722 | 0.3821 0.0121 m 0.0119 m he Root Mean Square Error (RMSE) is size statistics from difference between predicted value and observed value. This is common used For evaluate accuracy a model or estimator. RMSE is calculated with take root square of the average difference square between predicted value and observed value. The Root Mean Square Error of the Logarithm (RMSELOG) is size statistics from difference between logarithm from predicted value and logarithm from observed value. This is also used For evaluate accuracy a model or estimator. RMSELOG is calculated with take root square of the average difference square between logarithm from predicted value and logarithm from observed value

The research paper provides a detailed account of the development of a functional virtual reality (VR) platform serving as a digital archive for Penataran Temple, utilizing 3D point cloud technology. The study amalgamates a variety of 3D point cloud data sources, such as survey photogrammetry conducted through a camera mounted on an Unmanned Aerial Vehicle (UAV), evaluation photogrammetry executed on specific sections of the Penataran Temple, and extraction of 3D data from concealed relief panels employing a structured algorithm within the image camera reflex lens to archive 2D images. The VR framework is devised to offer two perspectives: a first-person viewpoint for detailed exploration of the cultural heritage, and an aerial perspective to enhance understanding of the comprehensive architectural layout of Penataran Temple situated in the Regency of Blitar, East Java.

The study involves the creation of a functional virtual reality (VR) system as a digital repository of Penataran Temple in Regency Blitar, East Java, Indonesia. This has been accomplished by utilizing 3D point cloud technology from various data sources. The system offers an immersive visual experience, improving the comprehension of the temple as a whole. Currently, research is in the initial stages of data collection, focusing on the main levels of the temple. However, there is an intention to conduct more complex 3D assessments at all levels of Penataran Temple and generate a highresolution VR representation encompassing the entire structure. Furthermore, the project will involve developing various applications, such as a web-based VR platform for remote viewing and a display technique that enables stereoscopic holography. The significance of developing this VR system lies in its substantial contribution to the preservation and interpretation of the cultural importance of Penataran Temple in Regency Blitar, East Java, with the potential for deployment across multiple platforms