We had in our project three main research goals. We accomplished all of them and produced additional findings connected to these main goals. The project fit within current discussions in the literature; therefore, we participated in the discussion and, consistent with these debates, extended our scientific goals. Below we describe the results of the initial project followed by the additional results of the project's extension.
Values play an increasingly important role in the social sciences. Today there is a general agreement in the literature that Schwartz’s (1992, 1994) theory is the best description of universal basic human values. Schwartz defines values as desirable, transsituational goals that serve as guiding principles in human life. The two key theses of Schwartz’s value theory can be formulated as follows: 1) All possible values can be subsumed under the headings of some more general value types, precisely described in the theory, and 2) the value types are structured around a circular motivational continuum.
In 2012, we published a paper on value theory refinement with Schwartz and colleagues (2012), which differentiated among 19 more narrowly defined values located, consistent with the theoretical expectations, in the circular structure. However, many open questions remained in the refined theory. We addressed some of these questions in our research project.
1. Interpretation of the method factor
Schwartz et al. (2012) differentiated among 19 more narrowly defined values measured by the new Portrait Value Questionnaire (PVQ-57). The differentiation of these values was tested using confirmatory factor analysis (CFA). Schwartz at al. (2012) included a method factor in the CFA that was unrelated to all of the value factors, while all value items loaded onto the method factor. Schwartz et al. (2012) argued that the method factor was introduced to control for social desirability and other types of measurement bias. In this project we explored empirically and more closely and meaning of this method factor, and investigated whether it could be interpreted as social desitability. To do this, we introduced external measures of social desirability into the CFA model and inspected the relationships between the common factor and the social desirability factors.
We analysed data from four studies conducted in Poland. We used three different measures of values: PVQ-40 (to measure 10 values), PVQ-57R (a revised questionnaire to measure 19 values), and the PVQ-57R-other (a questionnaire designed to measure 19 values rated by observers). Two different measures of social desirability were used: the Social Desirability Scale (Stoeber, 2011) and the scales for moralistic and egoistic response bias from the Big Five Questionnaire (Caprara, Barbaranelli, Borgogni, Vecchione, 2007).
Regression coefficients of the method factor on social desirability measured by external instruments were statistically significant and considerable only when values were measured using the traditional self-rating procedure. The effects disappeared when values were assessed using other ratings. Thus, the results supported the interpretation of the method factor as social desirability.
2. Measurement invariance of the PVQ-57
Establishing measurement invariance is a precondition for any meaningful cross-group comparisons. In the literature there is ample documentation of the well-known problems with measurement invariance of the PVQ-21 which was implemented in the European Social Survey (ESS). For instance, the analyses performed by Davidov, Schmidt and Schwartz (2008) supported configural and metric measurement invariance for only seven values and failed to support scalar invariance. The important question which came up after the development of the PVQ-5x was whether this new measurement instrument could reach higher level of measurement invariance than the previous one.
Measurement invariance is a topic that has received increasing attention in recent years. On the one hand, establishing measurement invariance is a necessary precondition for meaningful cross-group comparisons. On the other hand, establishing higher levels of measurement invariance and, especially, scalar invariance is a very demanding task that is very seldom successfully performed. New procedures have recently been proposed in the literature for addressing these difficulties of measurement invariance testing. Some of the procedures used different assumptions (e.g., the categorical multigroup CFA - MCCFA, Davidov et al., 2011), or looked for major local misspecifications (Saris et al., 2009). Others relaxed some of the more restrictive assumptions underlying the classical test of measurement invariance. Examples include Bayesian structural equation modelling (BSEM; Muthén & Asparouhov, 2013), exploratory structural equation modeling (ESEM; Marsh et al., 2009) and the Alignment procedure (Asparouhov & Muthén, 2013).
Exploring the measurement invariance properties of the value measurements further has become a major research topic in our research project. We analyzed the original and the revised PVQ scales using classical and new approaches. The main results are the following:
2.1. Using a categorical MGCFA (MCCFA) and a Baysian approach with the new PVQ-57 across nine countries, we established configural measurement invariance for all values, metric or partial metric invariance for almost all countries, and full or partial scalar invariance for approximately half of the new differentiated values. Thus, the cross-national invariance properties were much better than those discovered for older measurements.
2.2. We applied the local misspecification approach proposed by Saris et al. (2009) and combined it with a categorical MGCFA analysis to test for measurement invariance of the PVQ questions across datasets collected using online and offline modes of data collection. The analysis revealed that all values except societal security were scalar invariant.
2.3. We applied the newly implemented BSEM approach on the PVQ questions and found that all values display configural, metric and scalar measurement invariance across samples collected using online and offline modes of data collection.
2.4. We wrote for journal Studia Psychologica a tutorial paper explaining how to conduct measurement invariance test using he software packages Amos and Mplus.
2.5. We are going to continue the collaboration with Bengt Muthén and Peter Schmidt initiated in this project and apply the very new Alignment procedure to test for measurement invariance across countries in two invited research projects. The projects address value-behaviour relations and value-emotions relations and are coordinated by Shalom Schwartz. In both projects, cross-country comparability is of great importance; thus, the measurement invariance issue is an important topic. We have been invited to conduct this analysis.
3. The reversal of universalism (UN) and benevolence (BE) in the circle of values
An analysis of data collected in the research project on the theory refinement reveals that contrary to the theoretical expectations, two types of values, universalism and benevolence, are reversed in the circle of values. Schwartz’s examination of the various data collected to- date with different measurement instrument led him to the conclusion that the reversal was rather seldom in data collected using the Schwartz Value Survey (SVS) and more often with PVQ data. He also observed that SVS data were collected before the year 2000, after which it was data collection increasingly used the PVQ. Thus, it was not clear whether the replacement of values in the circle was a result of a change in value structure in the last few years or due to the instrument used to measure values (SVS vs. PVQ).
To answer the research question, we collaborated with the Psychometrics Centre at Cambridge University, which provided us with new SVS data, collected last year. We conducted a multidimensional scaling analysis on both row and centred data. In both analyses, we found no replacement in the SVS data. These findings support the hypothesis that the method (type of questionnaire) is a cause of the replacement.
4. Additionally, we extended the project by investigating the presence of the new values in data collected with instruments developed to measure (10) old value types. We argued that it should be possible to find some evidence for the refined theory using older data. We analyzed data collected in 13 countries (provided by Shalom Schwartz) and found that it is possible to differentiate more than 10 values using the PVQ-40 data, consistent with the refined theory.
5. In the refined theory, values form three levels of a hierarchy. Values on higher levels consist of values on lower levels and are defined by them. The hierarchical structure can be tested using confirmatory factor analysis (CFA) although no study to-date has conducted this test. We filled this gap and tested a model in which the 19 facets of values comprised the first order of analysis, basic values comprised the second order, and four higher order values comprised the third order of analysis. We used data collected in nine countries partly by Shalom Schwartz and other collaborators and partly by ourselves. The findings support the proposition that the more narrowly defined values in the refined value theory are subdimensions of more broadly defined values postulated in previous versions of the theory.
6. We also applied advanced methods of data analysis used in the human values project to the measurement of the Big Five personality traits within the framework proposed by Costa and McCrae (2008) in the field of personality psychology. We analyzed the hierarchical structure of personality traits in 33 countries (the data were provided by our collaborators at the Psychometrics Centre, Cambridge University). Results showed that a two-level structure, consisting of two higher order personality factors and five factors at the first order, was supported by the data in almost all countries.
