Pilot Study on User Experience Analysis of Universitas Indonesia Library Website

Academic libraries need to develop information and communication technology-based library services through library websites. The current library website does not attach importance to the information and document aspects in its presentation which should be made more artistic by using a modern user interface design (Ayyanar, 2021). The concept of modern user interface design cannot be separated from the importance of information architecture in understanding, finding information, and helping the presentation of information to be more organised on a website (Rosenfeld et al., 2015). Research related to information architecture in university libraries has been carried out by Ratnasari (2013) in a case study at the Open University Digital Library. Ratnasari (2013) found that library websites still do not meet the basic component principles of information architecture based on organisational systems, labelling systems, navigation systems, and search systems. In line with Ratnasari (2013), Utami (2017) found that the application of information architecture on the Indonesia Heritage Library website is still not appropriate and meets the principles of information architecture. Most of the discrepancies are found in the components of the navigation system and search system. This causes the Open University Digital Library and the Indonesia Heritage Library to require website development to support the optimization of library information architecture principles that can be viewed from the user experience. Research related to user experience on library websites has been carried out by Hidayat (2017) in his research related to user experience on the website of the National Library of the Republic of Indonesia. Hidayat (2017) found that National Library had implemented the principle of good user experience by looking at user needs and organizational needs. This is in line with the function of the National Library of Indonesia as a provider of information sources for the public interest where a good user experience can attract users to use the library website. Furthermore, Chatterjee & Kar (2018) state that success in the development of information technology lies in the active participation of users. In implementing active user participation, developers must create systems that are attractive and can gain user trust, such as security and privacy, as well as ease of obtaining information (Chatterjee & Kar, 2018). However, in making the user experience, it is necessary to pay attention to several aspects. Zarour & Alharbi (2017) explain that there are aspects in user experience research, namely from the side of user needs, they can be divided into pragmatic aspects (technology) and hedonic aspects (brands). Marketing and business communication between users and organizations, including aspects of the brand. In user experience design, the information architecture consists of a hierarchy that contains groups and subgroups for content and labels, and metadata, which contains structured data sets that are attached to individual items (Andrews, 2022). Dandy & Mayesti (2021) in their research related to the analysis of the information architecture of the Universitas Indonesia Library website (UI Library), explained that the application of the UI Library's website information architecture was in accordance with the theory put forward. The UI library implements top-down visualization on the main web page of the website, and bottom-up on the library collection web page. However, the research conducted by Dandy & Mayesti (2021) focuses on the point of view of librarians and researchers, using interviews and website observations. Additionally, Alshaheen & Tang (2022) in their research on 28 National Library websites from 15 countries in the world provides a new perspective on user experience and information architecture. His research aims to measure the quality of user experience and information architecture of National Library websites in the world. The results of his research stated that there were significant differences based on gender, academic status, and previous experience in using the National Library website. This refers to the National Library's website design must consider different user groups, so it is important to create a neutral design. In website development, National Libraries are advised to consider factors such as satisfaction and information quality (Alshaheen & Tang, 2022). Research related to library user experience is still relatively rare, especially for university libraries. Indeed, researchers have yet to identify a relationship between user experience and the information architecture of library websites. Most research has focused only on information architecture where the development of website user experience plays an important role (2021). Based on this background, the researcher explored the relationship between user experience and the perception of information architecture.

Quantitative research is a research approach that serves to test theoretical objectives by examining the relationship between variables. In user experience research, surveys are a method to obtain attitude data from users. This study uses random sampling as a sampling technique with a margin of error of 10% for a 95% confidence level. The research sample collected was 127 respondents who were undergraduate students of Universitas Indonesia and have used the Universitas Indonesia library website (lib.ui.ac.id) to search for information, and library collections. The research data collection was carried out online from February 28, 2022, to March 25, 2022. The survey was distributed using the Survey Monkey platform. Furthermore, the survey statement was prepared by combining PSSUQ, UMUX, and SUPR-Q which are the standard questionnaires in user experience research. Additionally, to combining the three questionnaires, the researcher also included a questionnaire in the Library Search UX research conducted by Stevenson & Larose (2016). The survey in this study used a scale of 4 with scale consisting of 1 = disagree, 2 = disagree, 3 = agree and 4 = strongly agree. Douven (2018) in Salasa et al. (2017) explains, the use of this scale is due to the characteristics of the Indonesian people who tend to choose a scale value that is closer to the midpoint and avoid the final scale value and this will cause a central tendency to bias. The reliability test for the entire questionnaire instrument showed Cronbach's alpha of 0.9, this figure indicates that the research questionnaire is valid. Furthermore, the analysis of the research results used the structural equation modelling – partial least square (SEM-PLS) method. Analysis of research results using the application tools SmartPLS 3.0, SPSS 26, and Notion. Furthermore, the research hypotheses are as follows: H1: Library search user experience significantly affects user perceptions of the information architecture of the Universitas Indonesia Library website. H2: Post-study system usability questionnaire (system quality (SysQual), information quality (InfoQual), interface quality (IntQual)) significantly affects user perceptions of the information architecture of the Universitas Indonesia Library website. H3: Standardized user experience percentile rank questionnaire (usability-navigation website, trust, loyalty, and user interface) significantly affects user perceptions of the information architecture of the Universitas Indonesia Library website. H4: Usability metric for user experience (usability-system capabilities) significantly influences user perceptions of the information architecture of the Universitas Indonesia Library website

Universitas Indonesia Library website (lib.ui.ac.id) is online public access catalog (OPAC) which serves to help users find information and collections in the library. The research respondents consisted of students from the Faculty of Humanities, Universitas Indonesia (UI). The aim of gathering their responses was to analyse the relationship between user experience and user perceptions of the information architecture of the Universitas Indonesia Library website. UI students have access to all services on the Universitas Indonesia Library website. The survey, which was distributed over a period of 25 days, obtained 127 respondents. The following is the demographic distribution of the respondents in this study: Research respondents were predominantly female with a total of 95 respondents (75%). Meanwhile, there were 32 male respondents (25%). Age demographics in this study were dominated by respondents who were 20 years with a total of 39 respondents (31%). In the semester distribution of respondents, the dominant respondents are in semester 6 with a total of 45 respondents (35%). Evaluation of the Outer Model. Evaluation of the outer model is done by testing the validity of the model that has been made. The validity measurement used consists of convergent validity and discriminant validity. Convergent validity analysis was determined using the loading factor and Average Variance Extracted (AVE) parameters. Abdillah & Hartono (2015) in Triasih (2021) explains if the loading factor value is > 0.7 and the AVE value is > 0.5, then the measurement can be categorized as having convergent validity. Discriminant validity was analysed through the cross-loading value of each variable. (Abdillah & Hartono, 2015) in Triasih (2021) explains if the cross-loading reaches > 0.7 then the measurement can be categorized as having discriminant validity. The value of convergent validity in this study can be seen from the value of outer loading on information architecture variables (A1-A4), library search UX (B1- B3), Post-study System Usability Questionnaire (C1-C12), Usability Metric for User Experience (D1 -D7), and Usability Metric for User Experience (E1-E4). Based on the calculation of the initial model, there are 10 indicators that have an outer loading value below the standard (< 0.7). Indicators that have an outer loading value below the standard, namely, B3, C5, C7, C8, C11, D2, D3, D5, E2, and E4, the ten indicators cause the AVE value in variables B, C, D, and E to get the same number. low, so the indicators are deleted. The research then recalculates the model after removing the 10 indicators, and produces a new outer loading value as shown in the path diagram below: The path diagram above shows that each value of the indicator already has a value limit according to the standard (> 0.7). This means that these indicators already have a value limit that can measure the variables properly. From a total of 30 indicators in the initial model, there are 20 indicators left for further testing. Here are the indicators that are omitted in the latest models: B3: I hope what I am looking for appears on the first page of the UI Library Website search results. C5: I find it easy to learn how to use the UI Library Website. C7 : If the UI Library Website gives an “error message,” the site tells me how to resolve the error. C8 : Whenever I make a mistake using the UI Library Website, I can easily and quickly solve it. C11: The UI Library Website has all the functionality and capabilities I would expect on such a site. D2: The information on the UI Library Website is credible. D3: The information on the UI Library Website is trustworthy. D5: I found another website more interesting. E2: I hope what I am looking for appears on the first page of the UI Library Website search results. E4: I spend a lot of time fixing things when using the UI Library Website. Next, the convergent validity test looks at the AVE value from the model calculation results. All variables in the model already have AVE > 0.5, which means that the AVE value of each variable is acceptable. This shows that more than 50% of the indicator variance can be explained. The information architecture has a 0.670 AVE value, which means 67% of the indicator variance can be explained. Meanwhile, the search UX library has the highest AVE value of 0.728, which means that 72.8% of the indicator variance can be explained. PSSUQ obtained a 0.649 AVE value, which means 64.9% of the indicator variance can be explained, SUPR-Q has an AVE value of 0.646 which means 64.6% of indicator variance can be explained, Finally, UMUX obtained an AVE value of 0.761 which means 76.1%. indicator variance can be explained. This means this research has met the specified standards. Furthermore, the measurement taken is discriminant validity by looking at the cross-loading value. All indicators that compose each variable in the study have met discriminant validity with a cross-loading value > 0.7 and have the highest correlation value compared to indicators with other variables. This means the measurements in this study were able to measure the research variables. Evaluation of the Inner Model The next measurement is the evaluation of the inner model. This evaluation serves to see the influence of the variables used in this study. Measurement of the inner model is done by looking at the value of R-square (R2). The R2 test looks at how much the value of the independent variable (B, C, D, E) affects the value of the dependent variable (A). Based on the results of the calculation of the R2 value, the resulting value is 0.711 where the independent variables, namely library search UX, post-study system usability questionnaire, standardized user experience percentile rank questionnaire, and usability metrics for user experience affect the dependent variable, namely the perception of information architecture by 71.1 % where the remaining 28.9% is influenced by other variables not measured in this study. Hypothesis Test Hypothesis testing is done to test whether the research hypothesis is accepted or rejected. Testing is done by using the bootstrapping function. Hypothesis testing is carried out based on the results of testing the inner model (structural model) which includes R2 output, parameter coefficients, and t-statistics (Triasih, 2021). The research hypothesis is accepted or rejected by looking at the significance of the t-statistics and p-values. The hypothesis is accepted if the p-values <0.05 or the t-statistics value> 1.96. The first hypothesis (H1) examines whether library search UX significantly affects the perception of the UI library's website information architecture. The test results show the value of t = 3.235 and p = 0.001, meaning that the t-statistic is significant because the value is > 1.96 and has p < 0.05. This explains that the first hypothesis is accepted. Based on the test results above, it can be concluded that the search UX library significantly affects user perceptions of the information architecture of the Universitas Indonesia Library website. The second hypothesis (H2) examines whether PSSUQ which consists of components of system quality (SysQual), information quality (InfoQual), and interface quality (IntQual) significantly affects the perception of the information architecture of the Universitas Indonesia Library website. The test results show the value of t = 1.843 and p = 0.066, this explains that the second hypothesis is rejected. Based on the test results above, it can be concluded that PSSUQ which consists of components of system quality (SysQual), information quality (InfoQual), and interface quality (IntQual) is not proven to significantly affect user perceptions of the information architecture of the Universitas Indonesia Library website. The third hypothesis (H3) examines whether SUPR-Q which consists of website usability-navigation, trust, loyalty, and user interface components significantly affects the perception of the UI library's website information architecture. The test results show the value of t = 3.587 and p = 0.000, this explains that the third hypothesis is accepted. Based on the test results above, it can be concluded that SUPR-Q which consists of website usability-navigation, trust, loyalty, and user interface components significantly affects user perceptions of the information architecture of the UI Library website. The fourth hypothesis (H4) examines whether UMUX which consists of usability-system capabilities components significantly affects the perception of the information architecture of the Universitas Indonesia Library website. The test results show the value of t = 2.499 and p = 0.013, this explains that the fourth hypothesis is accepted. Based on the test results above, it can be concluded that the usability metric for user experience which consists of the usability-system capabilities component significantly affects user perceptions of the information architecture of the UI Library website. Predictive Relevance and Model Fit Additionally, to testing the value of R2, in testing the model, a Q-square test (Q²) or predictive relevance is also carried out. The Q² test is carried out using the blindfolding function. The test results show a Q² value of 0.461, meaning that the model can be said to have predictive relevance. Ghozali & Latan (2015) in Triasih (2021) explains where the model is said to have predictive relevance if the Q² value > 0, on the other hand, if the Q² value < 0 then the model lacks predictive relevance. In addition, the research model can be said to be fit or good. This is indicated by calculating the fit model which shows the model's NFI (Normed-Fit Index) value of 0.71 where the model used in the study 71% is fit or good. Main Findings This study examines the relationship between user experience and perceptions of the information architecture of the Universitas Indonesia Library website (UI Library). The measurement of this study used a post-study system usability questionnaire (PSSUQ), usability metric for user experience (UMUX), and standardized user experience percentile rank questionnaire (SUPR-Q) (Sauro & Lewis, 2016), and library search UX from Stevenson & Larose (2016). PSSUQ is used to measure system quality (SysQual), information quality (InfoQual), and interface quality (IntQual). System quality (SysQual) serves to measure the quality of a computer system or application. Information quality (InfoQual) serves to measure the quality of the information provided by the system. Furthermore, interface quality (IntQual) serves to measure the display quality of computer systems/applications (Sauro & Lewis, 2016). The four statements in the PSSUQ that function to analyse the quality of information are then used as statements for information architecture variables. Based on the research results, system quality, information quality, and interface quality have the lowest path coefficient value of 0.199. These three aspects still have a positive influence on user perceptions of the information architecture of the UI Library website. However, there is a need for improvement in relation to the perception of information architecture. Furthermore, the questionnaire used is a usability metric for user experience (UMUX) to measure the usability of the Universitas Indonesia Library website on the system capabilities of the website, by looking at efficiency, effectiveness, and satisfaction with computer systems/applications (Sauro & Lewis, 2016). Based on the results of the study, the system capabilities aspect of usability has a path coefficient value of 0.214. This explains that website system capabilities have a positive influence on user perceptions of the UI library's website information architecture. Good system capabilities have an influence on the perception of the UI Library website's information architecture. This shows, that if the website has good system capabilities, then the perception of the UI Library website's information architecture can be said to be good. The researcher also used the SUPR-Q questionnaire to measure the usability of the UI Library website on the website navigation. This questionnaire also measures the level of trust, loyalty, and user interface of the website (Sauro & Lewis, 2016). Based on the results of the study, SUPR-Q has the highest path coefficient value compared to other indicators, namely 0.355. SUPR-Q has a positive influence on user perceptions of the information architecture of the UI Library website with a high level of significance. This shows that the website usability-navigation, trust, loyalty, and user interface systems have a relationship with each other in the perception of information architecture. This shows, that if the website has a good website usability- navigation, trust, loyalty, and user interface, then the perception of the UI Library website's information architecture can be said to be good. Lastly, the researcher used the library search UX questionnaire to measure the user experience performance of the library search engine (Stevenson & Larose, 2016). The results show that the search UX library has a path coefficient value of 0.227 whereas the UI Library website search engine has a positive influence on the perception of information architecture. Therefore, if the website search engine is good, then the perception of the UI Library website's information architecture can be said to be good. Based on the results of the overall analysis, the UI Library has good user experience performance and information architecture perception and can meet the needs of users in obtaining the information they need on the UI Library website. Implications of User Experience on Information Architecture This study implies that the user experience performance of the UI Library website is good in terms of library search, system capabilities, website navigation, trust, loyalty, and user interface. This aspect also has a strong relationship with the user's perception of the information architecture of the UI Library website. In addition, information quality, interface quality, and system quality have poor performance and need to be improved. Based on research conducted by Dandy & Mayesti (2021) the UI Library website architecture was created to have a reason behind its preparation. For example, the order of the online database on the lib.ui.ac.id page which is used for promotion is sorted by the most recent database. The text on the UI Library website explains the intent of the images or available information. When viewed as a whole, the preparation of the information architecture of the UI Library website functions to make it easier for users to understand and find the information that users need (Dandy & Mayesti, 2021). This study found that users agree that the displayed information, such as information retrieval, information effectiveness, and information organisation of the UI Library website helps users find what users need. However, there are some notes that need to be improved to optimize the user experience and perception of the UI Library website's information architecture. In analyzing the UI Library website navigation system with the perception of information architecture, the research is in line with research from Dandy & Mayesti (2021) which explains that the UI Library website uses textual and image formats in its navigation system. This research reveals that users get a good and easy experience in navigating the UI Library website where the UI Library website navigation system can be maintained or developed.

This study analyses user experience factors on the perception of information architecture on the website of the Universitas Indonesia Library (UI Library) using PSSUQ, UMUX, SUPR-Q and library search UX as measurement methods. The results showed that aspects of the UMUX, SUPR-Q, and library search instruments significantly influenced user perceptions of the UI Library website architecture. That is, how users see system capabilities, website navigation will affect user assessment of the information architecture of the UI library website. This also applies to the level of user trust and loyalty which will affect the user's assessment of the information architecture. The indicators in the three instruments are good, then the user's perception of the information architecture of the website can be categorized as good. In addition, aspects of the PSSUQ instrument were not proven to significantly affect user perceptions of information architecture. That is, how the user sees the system quality and capabilities, information quality does not affect the user's assessment of the information architecture of the UI library website. If the indicators in the instrument are good, it is not necessarily the information architecture of the website that can be categorized as good. In addition, the results of user experience affect 71.1% of perceptions of the perception of the information architecture of the UI Library website. In addition, descriptive analysis shows that the information organisation of the UI library website is the best indicator of information architecture, the search feature on the UI Library website is considered important for respondents to learn, and the UI library does not tell users how to resolve errors when used, the available information on the UI Library website you can trust, the UI library website isn't frustrating, but it still does not meet the user expectation. Furthermore, the path coefficients of each variable indicate that system quality, information quality, and interface quality have low values but have a positive effect on perceptions of information architecture. This explains that system quality, information quality, and interface quality require further development on the UI Library website. System capabilities and library search UX show quite satisfactory results in their influence on the perception of information architecture where both aspects can be maintained but still need further development. Website navigation, trust, loyalty, and user interface have the highest path coefficients. This explains that the UI Library website navigation system is good, but there is a need to improve the quality of the system, information, and interface. In general, Universitas Indonesia students trust and are loyal to using the UI Library website. This study also found that the SUPR-Q, UMUX, PSSUQ, and library search UX instruments can be applied to user experience research. The indicators in the instrument can be used to assess whether the user experience and perception of the website's information architecture can be said to be good and which indicators should be developed in website development. In this study, indicators that can be developed are system quality, information quality, and interface quality on the UI Library website. This research can be the basis for further research to examine other websites at different locations, times, and respondents.