Category Archives: Blog

Remote administration of the Cognitive Process Profile (CPP) assessment

By Marlouise Ferreira on March 24, 2020

 

The Cognitive Process Profile (CPP) is an advanced computerised assessment technique that externalises and tracks thinking processes to indicate a person’s cognitive preferences and capabilities.

To ensure conducive and standardised assessment conditions for the completion of the CPP, the assessment needs to be completed under the supervision of an accredited CPP practitioner. However, there are certain instances where in-person administration and supervision, which is preferred, may not be possible. This is where remote supervision and administration comes in handy.

Remote supervision can take place via a video-conferencing application such as Skype and Skype for Business, Zoom, Google Hangouts, etc.

This blog article outlines the process for setting up a remote assessment as well as for administering the CPP assessment when supervision takes place remotely.

 

The process to be followed when setting up a remote assessment:

Here is a high-level outline of the process that an accredited CPP practitioner can follow:

  • Send the candidate an email providing information and details on the CPP assessment beforehand, attaching the CPP “data release consent form” to be completed and signed by the candidate. An example of such an email can be found below:

 

Hi there,

I hope this email finds you well.

This email concerns the CPP assessment that you will be completing for Company X. Please be kind enough to check each point below in advance. The CPP is a supervised assessment and a facilitator will join you at the time of the assessment to assist you via Skype, Zoom or Google Hangouts.

What is the CPP?

The CPP, or the Cognitive Process Profile (CPP) is a computerised assessment technique which tracks a person’s thinking processes to determine their thinking skills and preferences. The results are used for HR purposes such as person-job matching, career pathing and development. The CPP tasks are unfamiliar. 

Preparation for the assessment

Could you please do the following as soon as you have time, in order to ensure that we have no problems on the morning of the assessment:

 1. Please sign and return the attached “data release form”. It is a consent form permitting us to assess you and to access your results.

2. Please ensure that you have a quiet place to complete the assessment.

3. Please ensure that you have a laptop or computer on which to complete the assessment and on which I can supervise you. (If you have a Mac (Apple) computer, please use Google Chrome for the assessment on Mac.) You may also need earphones and an external mouse.

4. Please make sure that you get a good night’s rest before the assessment. If you are ill or not feeling well, it is advisable not to do the assessment. It is always possible to reschedule.

5. Please put the following link into Google Chrome to see if you have all the required software for the assessment. (Please note that all the ticks should be green. If you do not have the necessary software installed on the computer, there will be a red X and you will need to download the necessary software) http://live.cognadev.com/cpp/compcheck.html

6. Please make sure that you have a stable internet connection (not a 3G connection) in order to complete your assessment.

7. Candidates may take 1 to 3 hours for the assessment as they work at their own pace. There is thus no time limitation for this assessment. You are, however, advised to set aside a minimum of 3 hours for your assessment.

8. Please indicate your availability to complete the CPP assessment. You can provide me with the dates and times that suit you. The CPP should preferably be done in the morning, ideally before 10:00 am.

 

Feel free to contact me should you have any questions.

Kind regards,

(Test administrator)

 

  • Set up the video conference meeting with the candidate by sending through a meeting invite to the candidate which contains the details to join in on the video conference call.
  • Create a CPP assessment nomination as one would normally do. Send through the assessment nomination email to the candidate on the morning of the assessment.

 

The process to be followed when administering the CPP assessment remotely:

On the day of the remote CPP assessment, a CPP accredited practitioner can follow these steps to ensure that the administration goes well:

  • Ask the candidate to activate their microphone and video camera. Also ask the candidate to switch off their cell phones and other devices, in order to limit disturbances or interruptions.
  • Ask the candidate to go to the assessment nomination which was sent through to them via email.
  • Instruct them to click on the blue ‘Start assessment’ button found at the bottom of the assessment nomination email. Should this button not work, have the candidate use the candidate.cognadev.com link to access the CPP assessment where they will then need to fill in the access key. Both the website link and the access key can be found below the ‘Start assessment’ button at the bottom of the nomination email.
  • When opening up the assessment, upon clicking on ‘Start assessment’, please confirm with the candidate that the assessment is opening in Google Chrome. If this is not the case, ask them to copy the link from the address bar and instead open it in Google Chrome, where they can then paste the link from the previous address bar. The reason being that Google Chrome saves the previously completed results should there be an internet connection problem.
  • Take the candidate through all of the preparation screens. This includes the following: 1) Tick that they are not a robot, 2) click on get started, 3) agree with the data release and proceed, 4) have the candidate fill in their personal information, 5) the candidate will also need to confirm their personal information by selecting OK.
  • The next screen should ask of the candidate to fill in the Supervisor Key. Either read out the Supervisor Key to the candidate or send the Key via the Skype message box (Test administrators can ensure that they have the Supervisor Key with them before commencing with the Skype session).
  • Take the candidate through the remaining preparation screens. This will include the following: 6) The selection of the game language and login screen, 7) the CPP compatibility analyser. With this analyser, the candidate will need to complete the soundcheck and confirm that all requirements are in place.
  • If the sound volume is too low, ask the candidate to adjust the volume level on their computer settings.
  • Once the candidate has continued, the CPP game will start loading.
  • Whilst the game loads, you can provide the candidate with a brief outline of the assessment. Thus, providing them with the test instructions.
  • The final preparation screen is the Candidate Questions. Go through the questions with the candidate. Focusing specifically on the first question which asks the candidate to agree that they have been informed of the possible consequences of doing the CPP. This question ensures that the candidate is doing and feeling fine on the assessment day.
  • Ask the candidate to tick off all of the other applicable questions.
  • Should the candidate be colour blind, please provide them with the aid for colour-blind candidates. This aid can be shared via the Skype message box.
  • Explain to the candidate that they can watch the instruction video to review the test instructions.
  • Once the candidate has gone through the instruction video, ask them to tick off that they have watched the video. Thereafter, the Start Game button will appear.
  • Before the candidate commences with the CPP, explain to them that for the duration of the assessment your own microphone and video camera will be switched off. The reason for you to switch off their microphone and video camera is to avoid disturbing them. However, ask the candidate to kindly leave on their microphone and video camera so that you can continue with supervising them.
  • If for some reason there are a lot of disturbances on the candidate’s end, kindly remind them that the CPP requires them to concentrate well; they really do need to complete the assessment in a quiet room or surroundings.

 

Should any test administrators have further questions or need technical assistance, please do not hesitate to contact us: info@cognadev.com or +27 11 884 0878.

Value Orientations and MBTI Types

By Paul Barrett and Maretha Prinsloo on February 25, 2020

©antishock / adobe.stock.com

 

Value Orientations

The Value Orientations (VO) reveals an individual’s worldviews, their assumptions about life and perceptual orientations. The VO draws – albeit not exclusively – from a body of knowledge (broadly referred to as “Spiral Dynamics”) generated by Clare Graves, refined and popularised by Don Beck and Chris Cowan, and critically discussed by various theorists (e.g. Ken Wilber).

Valuing systems act as catalysts in determining how people can be expected to apply their intellect, talents and personality. By identifying the value orientations of individuals one is likely to predict those in the workplace that are best suited to challenges such as: doing routine work in a team context; relentlessly pursuing goals under difficult circumstances; maintaining the status quo by ensuring quality and controlling risk in a reliable and structured manner; showing a customer orientation; flexibly generating new strategies, and persuasively creating value for all stakeholders; building relationships, developing others and creating harmony at work; as well as integrating and understanding the dynamics of the whole system to identity the leverage necessary to initiate change. The Spiral Dynamics (SD) model thus offers practical utility within the context of talent management.

Graves simplified the model by allocating different colours to each of the valuing systems of the SD model, in terms of which the behaviour of individuals, organizations, nations or any other socio-cultural group can be understood.

The various valuing systems involved, otherwise referred to as worldviews, organizing frameworks, belief structures, mindsets, perceptual filters, memes or decision-making frames, represent a continuously unfolding spectrum of consciousness and awareness. Each increasingly inclusive, consecutive sub-system or value orientation, integrates and transcends previous valuing systems. The entire spectrum involves a soft and dynamic hierarchy, also referred to as a holon. The seven levels depicted by the SD model below are organized around eight broad themes.

Below is a rather cryptic description of each of the seven valuing systems that forms part of the SD model of consciousness development (the spiral starts with the Beige valuing system which is usually associated with harsh life conditions where the challenge is to physically survive from day to day (the VO assessment does not measure this level as the VO is aimed at working populations who do not have to cope with these challenges).

 

The MBTI

The Myers-Briggs Type Indicator (MBTI) is an indicator of personality preferences in terms of four dichotomies. Individuals tend to use both sides of each pair; however, one is generally a natural preference. These are as follows:

From Wikipedia:

“Jung’s typological model regards psychological type as similar to left or right handedness: people are either born with, or develop, certain preferred ways of perceiving and deciding. The MBTI sorts some of these psychological differences into four opposite pairs, or “dichotomies”, with a resulting 16 possible psychological types. None of these types is “better” or “worse”; however, Briggs and Myers theorized that people innately “prefer” one overall combination of type differences. In the same way that writing with the left hand is difficult for a right-hander, so people tend to find using their opposite psychological preferences more difficult, though they can become more proficient (and therefore behaviorally flexible) with practice and development.

The 16 types are typically referred to by an abbreviation of four letters — the initial letters of each of their four type preferences (except in the case of intuition, which uses the abbreviation “N” to distinguish it from introversion). For instance:

  • ESTJ: extraversion (E), sensing (S), thinking (T), judgment (J)
  • INFP: introversion (I), intuition (N), feeling (F), perception (P)

These abbreviations are applied to all 16 types.

 

Accepted Value Orientations associated with MBTI types

The orientations which respondents indicate as being a positive expression of their world view/outlook are referred to as “Accepted Orientations”. Within the VO report, up to three of these values are reported upon for any individual.

The data from which the graph below was constructed is a sample of 441 heterogeneous-nationality incumbent employees who completed both the VO and MBTI; 388 internationals working for a large international corporate, and 53 South Africans working within South Africa for two smaller organizations. The incumbent job-roles span across senior management and C-suite executive roles, within industrial-production, infrastructure and security job-sectors. All the employees are graduates, including many with postgraduate qualifications. Not all employees completed all the assessments.

When considering specific clusters within the MBTI assessment in relationship to VO perceptual framework acceptance, it can be inferred that NT combination is mostly associated with Orange and Red value orientations. Additionally, it is also associated with the Yellow framework. This is likely due to the independent, realistic, technical but conceptual approach to ideas, information and people. The ST combination seems to be mostly associated with Blue value orientation suggesting a focus on tangible, practical and ordered information. NF profile seems to be in closer relationship to the collectivistic value orientations – Green and Purple – indicating their warm, somewhat relativistic and accommodating orientation.

Empirically, in order to show the frequencies of cases with particular value orientations who were assigned particular MBTI types, we use an alluvial plot format. This enables the visualisation of structural changes (here frequencies of cases) which can be viewed between two or more categorical or ordered-class attributes/variables. Although used more in network and time-flow applications, alluvial plot analytics are also useful for the kind of data available to us here.

What we see above is the ‘flow’ or pattern of frequencies of cases who prefer a particular value orientation, associated with their assigned MBTI type. The width of the path between an orientation and a particular MBTI type represents a higher frequency of cases in that pathway.

Looking at the figure above, we see that for this particular sample of educated business executives:

ENTP is most closely associated with Orange value orientation. Individuals who accept Orange orientation and present ENTP profiles tend to show high levels of energy in terms of seeking out new possibilities and challenges. Creation of ideas and generation of new solutions to difficult problems are likely to be stimulating to them. Perceptions are important to them hence they are likely to be perceptive of other people’s attitudes and even use these perceptions to get buy-ins from others.

ENTJ is associated with both Orange and Red value orientations. Individuals with such a combination are likely to show high levels of drive and energy for complex problems and achievement of goals and results. They may appear impatient when it comes to other people’s complacency and could come to decisions prematurely, without sufficient consideration of detail.

INTJ and ESTP are mostly associated with Red value orientation, with possible elements of the Orange value orientation. Individuals with such a combination of results can appear single-minded in terms of their attention to goals. They could appear action-oriented, realistic, critical and have high expectations of performance.

The essence of the relationship between ESTJ, ISTJ profiles and Blue value orientations seems to be a pragmatic, reliable and dutiful approach. These individuals tend to value procedures in place and will most likely appear thorough, hard-working and practical when dealing with problems or people.

The INTP profile is mostly associated with the Yellow value orientation. Individuals presenting with such a profile tend to be intrinsically interested in theoretical and intellectual problems. At times they may appear withdrawn or quiet, however, they present intellectual curiosity and are likely to become energetic when dealing with a topic which interests them.

ENFP profile seems to be associated with the Green value orientation. Such a combination is marked by relativism as individuals tend to see different perspectives and at times find it hard to decide on one point. They show concern for people and a  desire to understand and accommodate others.

 

In Conclusion

As shown above, the assessment of an individual’s value orientations, their worldviews and their assumptions about life and perceptual orientations adds a substantive and complementary perspective to the conversations which ensue when elaborating upon the contextual implications of a person’s MBTI type. It’s an interesting and perhaps thought-provoking finding.

The relationship between the actual and the CPP predicted level of work of employees

By Paul Barrett and Maretha Prinsloo on March 5, 2020

©mandritoiu/ adobe.stock.com

 

HR practitioners tend to expect a high degree of correspondence between the organization’s actual, assigned level of work of an employee and the individual’s current and potential levels of work as indicated by the Cognitive Process Profile (CPP).

This may, however, not necessarily be the case. The presence and magnitude of the mismatch between the actual and CPP predicted levels of work either suggests measurement error or may be due to factors external to the assessment itself. This blog focuses on external reasons for possible discrepancies between the person’s actual level of work complexity versus their cognitive competence.

Reasonable speculations regarding person-job mismatching may include that of: poor talent management decisions; placement based on networking connections; perceptions of individuals based on their reputations; nepotistic practices; affirmative action legislation such as quota systems; unequal educational opportunity and qualification; gender- and age-related factors; and/or normative cultural practices within an organisation or country, all of which may affect and skew the expected relationship between the level of work at which a person has been appointed, and their cognitive capability.

Given a substantive database of relevant biodata from individuals who have completed a CPP (over a quarter of a million cases), we are in a unique position to evaluate the prevalence of inappropriate level of work placement in the corporate environment, as well as speculate about its potential impact on workplace productivity and the strategic viability of organisations.

Our database includes a variable indexing the level of education for an individual, their current position (organized into meaningful job-roles/levels of seniority), and the functional area in which an individual is currently working.

In this blog article we report on the extent to which the level of education of an individual determines their current job-role or position, and the functional area they work within. The attained educational level is assumed to be a proxy for knowledge and intellectual skill and can therefore be expected to be associated with the job-role/job-level a person attains in life. If reasons other than education, qualification, and skills are resulting in people attaining job-roles/positions they would otherwise not attain, then we might be able to detect such anomalies using an appropriate visualisation methodology. Such evidence would help explain the occasional mismatch between the actual and CPP predicted levels of work.

Given we are working with class-category data and frequencies, it is helpful to visualize the data as the ‘flow-lines’, in an alluvial plot, which enables the visualisation of structural changes between categories (here frequencies of cases). The ‘flows’ between categories represent the frequencies of cases at a particular education level, where those cases distribute themselves within the current position categories, and in turn, where they distribute themselves in a functional work area. The width of the path between the three blocks of attributes (highest level of education, current position, and functional area) represents a higher frequency of cases in that pathway.

Figures 1 and 2 show the trends within the data.

Figure 1: The frequencies of cases at a level of education, their current positions, and in which functional area they now work

 

Figure 2: The frequencies of cases at a level of education, their current positions, and a subset of functional areas in which they now work

What we see from the above figures is that those who left school with no diplomas, certificates, or degrees seem to be represented at all job-levels and specialities (current position), but even more surprisingly, the bulk of them attain a leadership position of Managing Director or CEO in an Accounting/Finance functional area. A position one might expect requires professional qualifications as well as experience.

While educational level is not necessarily a direct causal variable for attaining such elevated positions in the work environment, the higher frequency pathway does suggest something ‘unusual’ is at work here, given the fewer numbers in that category for people with degrees. On the surface, it looks like some test-takers have been appointed to job-roles/positions for which they are academically/professionally unqualified.

This result would also explain why in some instances, the cognitive/performance-based CPP level of work indicator is at odds with the actual level of work of a test-taker. The logic here is that the CPP is a complex assessment of cognitive/thinking skills and styles; with academic study and qualifications being a proxy for the cognitive complexity required to undertake and successfully sustain such study.

We would expect people (on average) to be at a disadvantage when it comes to attaining job roles which require specific knowledge and skills, or senior roles where good judgement, the capability to think clearly on a range of topics, and strategic thinking is definitive of the role (as in Managing Directors and CEOs). Yet the data does not reveal such a clear picture.

Does any of this matter?

Well, yes. Because if individuals are being appointed to specialist, senior management and/or leadership job roles on the basis of attributes which take precedence over their demonstrated expertise and intellectual performance, then over time the consequences might prove to be disastrous for any organization, depending upon the ‘protective’ qualities of individuals surrounding the misplaced individual.

The practical impact of failing to match people capabilities and job requirements are also suggested by the findings of this study where the majority of candidates with lower educational levels; namely, 10- and 12-year schooling, certificates and diplomas, were resident in and assessed in South Africa. Given that many of these candidates are employed in management and executive roles in the public sector and/or state-owned entities (SOEs), the disintegration of the services within these sectors over the past two decades may not be purely coincidental and seems to have contributed to adverse implications at a national level.

Using self-report personality questionnaires for high-stakes employee assessment: The end of an era?

By Paul Barrett on January 15, 2020

©mrr / adobe.stock.com

 

I know, ‘end of an era’ looks ridiculous at first sight, given the huge uptake of these questionnaires in the assessment market. From a recent overview of the area:

“Personality testing is a big business. It has an estimated market value of £4 billion. There are now hundreds of vendors distributing and selling an array of tests (1,319 at the last count), deployed across a range of applications” (Munro/Envisia Learning, 2019).

But the author of this overview begins his 49-page evidence-based investigation with:

“The proposal is that self-report personality measures have now had the best part of a century to demonstrate their practical value in employee selection. But their initial potential – despite some promising signs – has not been translated into the kind of performance that has had a significant organisational impact.”

And concludes:

“We can only anticipate another century of counter-productive debate and confusing claims in which self-report personality test data from applicants account for less than 5% of work performance.”

This conclusion is given even more credence by the results presented in a 2016 working paper (now submitted for publication) from Frank Schmidt and colleagues entitled: “The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings” This paper updates the famous 1998 article from Schmidt & Hunter (which summarised the results from 85 years of publications). In table 1, p. 65 of the 2016 article, we see that personality assessment, including emotional intelligence and standard situational judgement tests, is of marginal predictive accuracy of job performance, compared to ability assessment, structured and unstructured interviews.

Table 1: Operational validities of personality attributes as predictors of Job Performance, extracted from Table 1, Schmidt, Oh, & Shaffer (2016)

And, similarly in the Annual Review of the area in 2014, by Neal Schmitt, who states:

“The relationships between personality measures and performance vary with which of the Big Five constructs one considers and appear to be generalizable only in the case of conscientiousness. Observed correlations between performance and individual measures of personality are almost always less than .20, and corrected correlations rarely exceed .25.” (p. 46)

These results are also compounded by the near-terminal problems associated with correcting correlations (disattenuation) for range restriction and unreliability in order to compute ‘operational’ validities (LeBreton, Scherer, & James, 2014; Roth, Huy, Oh, Van Iddekinge, & Robbins, 2017; Johnson, Deary, and Bouchard, Jr., 2018).

Furthermore, after almost 80 years of self-report assessment of personality, accompanied by developments in psychometric test theory including ideal-point and forced-choice ipsative-to-normative Thurstonian item response theory (IRT) upon which the SHL OPQ- “Reimagined” is based, what we find is that none of it has had any enduring/substantive impact except on the marketing and sales of tests. Indeed, two recent articles show that the whizz-bang personality ideal-point computer-adaptive administration methodology along with SHL’s Thurstonian IRT scoring of the OPQ are no better/more predictively accurate than using a simple item sum-score (Oswald, Shaw, and Farmer, 2015; Walton, Cherkasova, & Roberts, 2019; Fisher, Robie, Christiansen, Speer, & Schneider, 2019).

This relative failure to show any real improvement in assessment is also confounded by the realisation that for many high-stakes, leadership, and high job-role-autonomy positions, such candidates invariably possess ATIC (Ability of candidates To Identify Criteria for a job-role), which is related to a candidate’s cognitive abilities and as a result increases the validities of job performance in such candidates. It’s not faking as many I/O psychologists would suggest or try to eliminate, but something which is indicative of perceptive reasoning and information extrapolation in candidates; attributes clearly of value to employers (Kleinmann, Ingold, Lievens, Jansen, Melchers, & König, 2011; Ingold, Kleinmann, König, Melchers, & van Iddekinge, 2015; Geiger, Olderbak, Sauter, & Wilheim, 2018).

And then, there is the changing workplace features of the modern corporate and job-roles within such corporates. As noted in an earlier Cognadev blog, what’s critical now for choosing a psychological assessment is an answer to our Question #3: What level of autonomy does the job-role possess? By autonomy, I mean the degree to which the employee will have the freedom to choose how they wish to achieve necessary work-goals, make decisions and ‘influence’ (in the widest possible meaning of that term) other employees and important/substantive organizational outcomes. Self-report information is not sufficiently trustworthy in this context because we need to know how a person actually functions cognitively when undertaking a complex novel task that requires information manipulation, integration, and extrapolation; and not how they think they function. Before an employer allows an employee considerable autonomy, they need to be assured it will be used responsibly.

Given the huge body of evidence presented in Andrew Munro’s article and the recent publications noted above, it might be concluded that those continuing to promote the famous commercial personality assessments for high-stakes assessment are selling products which now appear to have relied for their success more upon marketing expertise than truly long-term substantive evidence of ‘effect’. The use of self-report questionnaire assessment for modern high-stakes job-roles is, as a result, gradually fading away year-on-year because there is simply no enduring, substantive evidence that anyone can point to as clear, unambiguous, evidence of ‘effect’.

Indeed, from 1996 up until 2012 (the last available year of this specific survey data), an analysis of annual US Harris Poll data on the public perceptions of leadership in a variety of large corporates and organizations revealed that as the ‘$$ spend’ on leadership training/development increased, so did public perceptions and confidence in leaders decrease (Barrett, 2012). It’s the same with the usual employee engagement assessment and interventions. For example, Forbes magazine in 2018 reported the results of a Gallup international survey on employee engagement, with an article entitled: Our approach to employee engagement is not working.

“A staggering 87% of employees worldwide are not engaged”.

 

Next Generation Assessment

With the increasing focus on the psychology of high-stakes candidates, more sensitive and performance-based information is required about a candidate’s psychology and cognitive functioning than is possible to acquire via self-reports or GMA assessments. Indeed, the simple strategy (utilised by some professional services organizations) of hiring the highest-scoring candidates on ability tests is now looking unwise, as Robert Sternberg pointed out in 2017 “The IQ of smart fools”, and in 2018: “Speculations on the role of successful intelligence in solving contemporary world problems”.

This “Next Generation” of psychological assessment requires a qualified psychologist to interpret assessment results in conjunction with other external information sources and job-role/organizational experts because now we are addressing a person’s cognitive capabilities, their preferred cognitive styles/biases of working with/manipulating information, and their personal values and motivations which influence their cognition. That interpretative expertise is not something that can be acquired in a 2-day test-publisher training course.

Whether selecting entrants for leadership development or C-Suite leaders, the goal now is to assess ‘in depth’ and not rely upon the cumulative summed responses to self-report questionnaire items. Indeed, some consulting psychologists simply use the items themselves as probes, asking candidates about why they chose a particular response; as that questioning can sometimes reveal far more about how a candidate is thinking than a simple cumulative sum score of such items. But, the more powerful/robust information comes from formally observing cognitive functioning rather than asking for self-reports of functioning; looking at candidate preferences for particular values and motivations without asking them rather transparent and simple single-statement questions as is the norm for typical values/interests/motivation questionnaires.

The overarching psychological model now used as a focus for interpretation is drawn from dynamical integral psychology, not discrete-attribute psychometrics.

An example of such an assessment approach is described in detail in a previous Cognadev blog. A more in-depth explanation of the theoretical foundation and the assessment approach can be found in Prinsloo & Barrett (2013). The methodological approach, and thus the assessment techniques, namely the Cognitive Process Profile (CPP) and the Learning Orientation Index (LOI), are primarily based on the self-contained, holonically organised Information Processing Model (IPM) of Prinsloo. The assessments involve automated simulation exercises by which thinking processes are operationalised, externalised and tracked across thousands of measurement points. A person’s performance is then analysed by algorithmically based expert systems. Extensive reports are automatically generated which indicate the test candidate’s stylistic preferences, information processing competencies (IPCs), cognitive complexity, learning potential/cognitive modifiability, developmental guidelines, levels of metacognitive awareness, and an ideal work environment. Aspects such as conceptual skills, logical capability, strategic orientation, judgement and decision making are specifically addressed as well.

The above approach, formulated in the early 1990’s, therefore does not rely on the typical psychometric assumption that ‘true scores’ can be located for specific ‘traits’. In fact, the previous century’s thinking enshrined in true-score psychometrics was finally laid to rest in 2002 by Borsboom and Mellenbergh.

Perhaps the obvious ‘performance-based’ model for assessment is the use of Assessment Centres. But these are expensive to set up and run in such a way that their validity is robust and maintained (as noted in Jackson, Michaelides, Dewberry, & Kim, 2016; Dewberry & Jackson, 2016). In many ways, though, the CPP and LOI can also be regarded as automated assessment centres.

Cognadev’s unique performance-based, holonic assessment of the integral triad of cognitive complexity + preference + learning agility, and the additional unique approach to values and motivation assessment, is now of increasing relevance. Two very recent blog articles explain why in some detail:
Cognitive complexity and cognitive styles: implications for strategic work
and, part 4 of a 4-part series on intellectual capital management:
Intellectual Capital Management: Assessment Products and Constructs

And they also make clear why interpretation of the acquired assessment information requires a psychologist; because the integration of the assessment information requires more knowledge and insight into human psychology than interpreting the typical generic narrative statements which appear in most computer-generated self-report questionnaire test reports.

This may appear to negate the objectivity inherent in the generation of attribute magnitudes, orders, and classes, but is merely a recognition of the fact that the integrated assessment of any individual’s cognitive functioning, motives, and values requires much more thought and insight than the computation of a few numbers as though we were making measurement of a physical science base or derived-unit quantity.

As Jan Smedslund (2009) began the first of a series of three articles:

“This article contains a comparison between what goes on in psychological research and in psychological practice. These two kinds of activities both start from a threefold common but unstated base, namely what we all know about being human because we are humans, what we know about each other because we participate in shared meaning systems (language and culture), and what we know about individuals in their individual life situations. From this common starting point, the two activities have developed differently. In order to help people in real life, practitioners have pursued a search for effectiveness, whereas researchers, in order to produce knowledge, have pursued a search for invariance (exact or probabilistic regularities). Traditionally, practitioners are supposed to learn from the results of research. In doing research one assembles evidence for and against hypotheses linking measured variables. The results very often take the form of small average differences and low correlations. These same variables are also taken to be involved in practical psychological work and, therefore, practitioners should be able to profit from reading the research reports. However, it is difficult to apply the scientific results, given the highly complex stream of persons, circumstances, and events that make up the practical experience. Therefore, it is frequently hard to see when and how practitioners can learn anything useful from the researchers.

Here, I argue that it is mostly the other way around and that researchers must listen to and learn from what goes on in practice. Practitioners are forced by their commitment to people in real life to take into account all our advance knowledge of psychological phenomena, whereas researchers, as I will argue, frequently have ignored, excluded, or circumvented much of this knowledge in order to produce invariant empirically based findings.” p. 778-779.

That’s why Cognadev assessments are so different and so important for high-stakes assessment. We form judgments and knowledge-claims about an individual based upon formally observed and computational-rule-based ‘scored’ novel task performance (resulting in attribute magnitudes, orders, and classes) as well as considering the additional complexity provided by the assessment of motivations and values acquired by algorithmically examining the judgments and preferences made by an individual. Finally, all this information is contextualised for specific work purposes by means of the Contextualised Competency Mapping (CCM) tool, with the final results integrated by a trained psychologist in the context of the job-role expectations and specific requirements of a client.

Our evidence-bases for the judgments we formulate are robust and open to third-party evaluation in our test manuals and online Technical Report Series, except we do not assume psychological attributes are physical science quantities like length and mass; as though a magnitude on an attribute like “Curiosity” can be interpreted in the same precise way as we would “interpret” a quantitative measure of length.

There are no shortcuts to getting it right in psychological assessment when we refuse to make the untested assumptions made by so many test publishers and academics concerning the quantitative structure of psychological attributes (Michell, 1997; 2008; 2012; McGrane & Maul, 2020). These assumptions have now been openly challenged in courts of law in NZ and the US, (Barrett, 2018; Beaujean, 2018).

As Michael Maraun put it in 1998:

“There is no public, normative status at all to assertions like ‘Tomorrow we are going to measure little Tommy’s dominance’. What does this mean? In contrast to the teaching of the use of concepts such as weight and height, the teaching of the use of concepts such as dominance and intelligence does not involve the teaching of rules for measuring. There is no common language standard of correctness for a claim like ‘I measured Sue’s leadership this morning’. In other words, there is no public, standardly taught notion of what it is to be correct in making such an assertion; instead, it sounds merely curious.” p. 455

That is why Cognadev approaches high-stakes assessment as psychologists and scientists; using tools, methods, analytics, and insights which are allied to assessing ‘effectiveness” realistically rather than pretending any of this can be made to look and sound like mechanical engineering or a branch of applied mathematics (Schönemann, 1994). As I have shown above that latter approach hasn’t worked and we now understand why; which is the reason why we don’t continue doing ‘same-old’ as though this will somehow achieve the elusive success which has failed to emerge over the previous decades of doing things the same way.

 

References

Barrett, P. (2012). The public perception of institutional leadership as a function of $$ spend on executive training and development. https://www.pbarrett.net/stratpapers/annual_spend_and_leadership_perceptions.pdf

Barrett, P.T. (2018). The EFPA test-review model: When good intentions meet a methodological thought disorder. Behavioural Sciences (https://www.mdpi.com/2076-328X/8/1/5 ), 8,1, 5, 1-22.

Beaujean, A.A., Benson, N.E., McGill, R.J., & Dombrowski, S.C. (2018). A misuse of IQ scores: Using the dual discrepancy/consistency model for identifying specific learning disabilities. Journal of Intelligence (http://www.mdpi.com/2079-3200/6/3/36 ), 6, 36, 1-25.

Borsboom, D., & Mellenbergh, G.J. (2002). True scores, latent variables, and constructs: a comment on Schmidt and Hunter. Intelligence, 30, 505-514.

Dewberry, C., & Jackson, D.J.R. (2016). The perceived nature and incidence of dysfunctional assessment center features and processes. International Journal of Selection and Assessment, 24, 2, 189-196.

Fisher, P.A., Robie, C., Christiansen, N.D., Speer, A.B., & Schneider, L. (2019). Criterion-related validity of forced-choice personality measures: A cautionary note regarding Thurstonian IRT versus Classical Test Theory scoring. Personnel Assessment and Decisions (https://scholarworks.bgsu.edu/pad/vol5/iss1/3/ ), 5, 1, 1-14.

Geiger, M., Olderbak, S., Sauter, R., & Wilheim, O. (2018). The “g” in Faking: Doublethink the validity of personality self-report measures for applicant selection. Frontiers in Psychology: Personality and Social Psychology (https://doi.org/10.3389/fpsyg.2018.02153 ), 9, 2153, 1-15.

Ingold, P.V., Kleinmann, M., König, C.J., Melchers, K.G., & van Iddekinge, C.H. (2015). Why do situational interviews predict job performance? The role of interviewees’ ability to identify criteria. Journal of Business and Psychology, 30, 2, 387-398.

Jackson, J.R., Michaelides, G., Dewberry, C., & Kim, Y.-J (2016). Everything that you have ever been told about assessment center ratings is confounded. Journal of Applied Psychology, 101, 7, 976-994.

Johnson, W., Deary, I.J., & Bouchard, Jr., T.J. (2018). Have standard formulas correcting correlations for range restriction been adequately tested? Minor sampling distribution quirks distort them. Educational and Psychological Measurement, 78, 6, 1021-1055.

Kleinmann, M., Ingold, P.V., Lievens, F., Jansen, A., Melchers, K.G., & König, C.J. (2011). A different look at why selection procedures work: The role of candidates’ ability to identify criteria. Organizational Psychology Review, 1, 2, 128-146.

LeBreton, J.M., Scherer, K.T., & James, L.R. (2014). Corrections for criterion reliability in validity generalization: A false prophet in a land of suspended judgment. Industrial and Organizational Psychology: Perspectives on Science and Practice, 7, 4, 478-500.

Maraun, M.D. (1998). Measurement as a normative practice: Implications of Wittgenstein’s philosophy for measurement in Psychology. Theory & Psychology, 8, 4, 435-461.

McGrane, J. A., & Maul, A. (2020). The human sciences, models and metrological mythology. Measurement (https://doi.org/10.1016/j.measurement.2019.107346), 152,107346, 1-9.

Michell, J. (1997). Quantitative science and the definition of measurement in Psychology. British Journal of Psychology, 88, 3, 355-383.

Michell, J. (2008). Is Psychometrics Pathological Science? Measurement: Interdisciplinary Research & Perspective, 6, 1, 7-24.

Michell, J. (2012). Alfred Binet and the concept of heterogeneous orders. Frontiers in Quantitative Psychology and Measurement, http://www.frontiersin.org/quantitative_psychology_and_measurement/10.3389/fpsyg.2012.00261/abstract, 3, 261, 1-8.

Munro, A. (2019). Personality testing in employee selection Challenges, controversies and future directions. Envisia Learning: https://amazure.envisialearning.com/wp-content/uploads/2019/12/Personality-Testing-Employee-Selection.pdf

Prinsloo, M. & Barrett, P. (2013). Cognition: Theory, measurement, implications. Integral Leadership Reviewhttp://integralleadershipreview.com/9270-cognition-theory-measurement-implications/

Roth, P.L., Huy, L., In-Sue, O., Van Iddekinge, C.H., & Robbins, S.B. (2017). Who r u? On the (in)accuracy of incumbent-based estimates of range restriction in criterion-related and differential validity research. Journal of Applied Psychology, 102, 5, 802-808.

Schmidt, F.L., & Hunter, J.E. (1998). The Validity and Utility of Selection Methods in Personnel Psychology: practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 2, 262-274.

Schmidt, F.L., Oh, I-S., & Shaffer, J.A. (2017). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings. Available on request from: https://www.researchgate.net/publication/309203898_The_Validity_and_Utility_of_Selection_Methods_in_Personnel_Psychology_Practical_and_Theoretical_Implications_of_100_Years_of_Research_Findings

Schmitt, N. (2014). Personality and cognitive ability as predictors of effective performance at work. Annual Review of Organizational Psychology and Organizational Behavior, 1, 45-65.

Schönemann, P. (1994). Measurement: The reasonable ineffectiveness of mathematics in the social sciences (Chapter 10, pp 149-160). In I. Borg and P. Mohler (Eds.). Trends and Perspectives in Empirical Social Research.

Sternberg, R.J. (2017). The IQ of smart fools. APA Observer (online: https://www.psychologicalscience.org/observer/the-iq-of-smart-fools ), December, , 1-4.

Sternberg, R.J. (2018). Speculations on the role of successful intelligence in solving contemporary world problems. Journal of Intelligence (http://www.mdpi.com/2079-3200/6/1/4 ), 6, 4, 1-10.

Walton, K.E., Cherkasova, L., & Roberts, R.D. (2019). On the validity of forced choice scores derived from the Thurstonian item response theory model. Assessment (https://doi.org/10.1177/1073191119843585), In press, 1-13.

CPP Career Group Analytics: Action research and Artificial Intelligence (AI) analyses

By Christoff Prinsloo and Maretha Prinsloo on December 24, 2019

© 腾龙 郭/ adobe.stock.com

 

Introduction

The goal of this study differs from that reported in part 2, which focused on the information processing tendencies of high level CPP performers or high capability groups across career fields. This study involved the use of AI techniques to identify the unique processing tendencies of various career groups, regardless of the levels of CPP performance involved. Both cognitive styles as well as Information Processing Competencies (IPCs) were considered to differentiate between groups. AI findings were also compared to typical career trends which emerged from qualitative action research by means of personal feedback on CPP results (and in some cases alternative values and personality test results as well) to approximately 5,000 individuals.

Given the challenges involved in statistically analyzing psychometric data, especially those of averaging effects, AI and action research are both well-positioned to reflect more realistic findings than is the case with “blind” number crunching.

For current purposes, an Artificial Intelligence (AI) product, namely Microsoft Power BI’s Key Influencer Visualization was applied to the CPP results of a sample of 24,600 candidates. Normalized-Standardized T-scores were used. The goal was to identify dimensions that best discriminate between various career groups by identifying “key influencers”. These differentiators indicate the likelihood of a specific career group to be characterized by certain processing tendencies at specific score ranges.

The analysis sample spanned several career fields, genders, age groups, geographical areas, educational levels and CPP results as indicated below:

 

Table 1: The Career groups used in this analysis

 

Figure 1: The gender representation of the sample

 

Figure 2: The sample’s educational level and educational field

Figure 3: The geographical areas represented

 

Findings

The AI program used in this study was aimed at analyzing the differences between groups to identify the cognitive dimensions that best differentiate a specific career group from all the others.  Whereas rank ordered scores (in the case of the Cognitive Styles) and Standardized T-scores (in the case of the IPCs) were used the first part of this blog series, rank ordered Normalized-Standardized T-scores (NST-scores) were used in part 2 and in this final part of the blog series.

The likelihood that a processing score (a specific factor) predicts the career group indicated, is also provided. For example, for older Accountants there is a likelihood of 4.19 for a factor value beyond the listed threshold value to belong to the career group of older Accountants. In other words, any candidate is 4.19 times more likely to belong to the older Accountants target group if they have an Analytical score above 28, which is the threshold value.

Another example, looking at the thinking styles of younger Accountants, shows the Intuitive Style ranking as the most powerful differentiator with a likelihood of 4.21. Of all the career groups, respondents with an Intuitive style score above 77 are thus 4.21 more likely to belong to the group of younger Accountants than of respondents with lower Intuitive Style scores. The bar chart below shows the distribution of respondents in the career groups for various levels of the specific factor.

For younger Accountants, the blue bar shows that 3.53 % of the entire sample belongs to the career group younger Accountants, whereas on average, 0.69% of the sample would have belonged to younger Accountants. That ratio (3.53/0.69) roughly indicates the 4.21 factor.

The styles and IPCs that best differentiate a specific career group from the others are not necessarily the group’s highest score dimensions, but may be the opposite or alternatively, reflect a specific score range.

Examples of the way in which the best differentiators per career group are indicated appear in Figure 4.

 

Figure 4: Best cognitive Style and IPC influencers of the older and younger Accountant groups

 

The results obtained through this AI study are reflected in Table 2 and Table 3:

Table 2: Key Cognitive Style differentiators of various career groups

 

Table 3: Key Information Processing Competencies (IPCs) of various Career groups

 

An interpretation of the findings of this AI study

Accountants

The key influencer for older Accountants, or the stylistic preference that best differentiates them from other career groups is that of the:

  • Analytical style (T62 above 28 at a factor of 4.19)

And the key Information Processing Competency (IPC) influencers for this career group are those of:

  • Metacognitive Awareness and Alertness (T34 above 22 – at infinity)
  • Spontaneous comparison of elements (T09, above 80 at a factor of 5.98)

Younger Accountants (24 – 44 years of age) are best differentiated from other career groups by the stylistic preference of the:

  • Intuitive (T65 above 77 at 4.21)
  • Memory (T66 above 69 at 2.37)

And the key IPC influencers those of:

  • Extracting core elements (T23 above 80 at 41,68)
  • Speed (T56 above 75 at 9.77)
  • Learning (T44 above 72 at 8.51)

This may imply that long term involvement in an accounting career encourages the use of a high degree of Metacognitive awareness and an Analytical approach as characterized by detailed, precision and rule-based tendencies.

In the case of younger Accountants who, more so than other career groups tend to show an Intuitive stylistic approach (which to some extent overlaps with a Learning style), there is a reliance on the Memory of their knowledge base to meet the detail requirements of their jobs. Over time their Intuitive approach may therefore make way for a more detailed, factual approach to problem solving and strategizing. However, the exceptional speed, flexibility and learning orientation of younger Accountants, combined with the skill of identifying and extracting core elements, ensure highly effective cognitive functioning that basically covers most of the processing bases.

Action research aimed at identifying the cognitive characteristics of younger Accountants has indicated that most show a Tactical Strategy with potential for Parallel Processing SST complexity level. More than 70% of younger Accountants show particularly well-developed Logical-Analytical skills and preferences. The Intuitive approach of younger accountants come as a surprise, but less so given the Memory component which supports both an intuitive and an analytical application.

 

Actuaries

The key stylistic differentiators seem those of the:

  • Holistic (T64 above 63 at 6.39)
  • Memory (T66 above 66 at 5.75)
  • Analytical (T62 above 64 at 5.67)
  • Low Impulsive / Reactive (T68 less than 35 at 5.59)
  • Low Random / Trial-and-error (T69 less than 32 at 5.31) approaches.

In terms of their IPCs they are best differentiated from other career groups in terms of:

  • Metacognitive strategies for Logical reasoning (T30 above 73 at 13.46)
  • General processing approach (T04 above 67 at 7.56)
  • The use of Hypotheses (T42 above 68 at 7.07)
  • Metacognitive Strategies (T37 above 60 at 6.30)
  • Metacognitive Strategies for Exploration (T06 above 64 at 6.24)
  • The tendency to Extract Core Elements (T23 above 67 at 6.18).

As in the previous studies, Actuaries again showed a highly balanced cognitive approach in that they seem metacognitively aware, do not resort to random and impulsive approaches and accommodate both detailed and holistic information. Action research of the cognitive tendencies of Actuaries has indicated that this career group tends to obtain higher scores than most other career groups on almost all the processing scores – particularly those of Rule orientation and Categorization – which is not typical of comparable professional fields. They thus apply their thinking in a most rigorous, detailed and structured manner in a way that exceeds any of the other career groups.

 

Finance (other) Group

In this somewhat diverse career group which include Bankers, Economists, Financial advisors, Bookkeepers, etcetera, the relatively weak AI identified stylistic influencers included:

  • Impulsive (T68 below 52 at 1.65)
  • Learning (T72 above 48 at 1.58)
  • Holistic (T64 above 50 at 1.57).

The IPC influencers were:

  • Coherence (T16 above 40 at 1.67)
  • Exploration (T05 above 30 at 1.64)
  • Card Movements (T00 67-81 at 1.64)

Given the diversity of this group, averaging effects have resulted in somewhat unremarkable results. The group as a whole do, however, seem to be characterized by somewhat uneconomical exploration processes which in turn tend to lower the effectiveness of their more complex processes.

Previous research studies of the preferences and capabilities of financial people other than Accountants, included empirical and qualitative explorations. These studies were conducted in various sectors within the Financial Industry, such as Banks and Insurance firms, as well as across geographical regions and specific jobs. Interesting results, for example, include that Economists at a Central Bank, who generally show a Tactical Strategy SST level or higher, and who have postgraduate or multiple degrees, showed either a surprisingly high Random (which is an ineffective approach) or a Holistic cognitive style (which is an effective big picture approach within both practical and theoretical contexts).

Action research also indicated that almost all Economists seem to show an ideas orientation which differs from the more factual and logical approaches which is generally characteristic of Accountants. Credit Managers from Commercial versus Corporate Banking divisions showed very clear differences in cognitive approach. In one such study, 79% of Credit managers from the Corporate division of a bank showed a marked “left-brain” preference through the application of Logical, Analytical and Reflective styles in dealing with issues of corporate viability and financial structuring. Credit managers from the Commercial division of the same bank, who mostly deal with individuals and small businesses, showed a typical “right-brain” approach though a reliance on Metaphoric, Integrative and Random approaches. The Commercial Credit Managers also showed a significantly higher need for structure than the Corporate Credit managers did. Insurers from various regions seem to work at different levels of complexity (SST): Insurance Executives from South African, the United Kingdom and one Asian insurer, seemed to work at similar levels of complexity which significantly differed from that of their Australian counterparts.

Those involved in IT banking systems from South Africa and the UK also showed relatively similar cognitive profiles which clearly differed from that of the rest of SADC African banks in terms of levels of complexity and stylistic preferences. Whereas the former group prefer a Logical and Integrative approach to problem solving, the latter SADC group applied a more operational approach characterized by Explorative and Reflective cognitive styles.

 

Administrators

The key stylistic influencer seems to be that of:

  • Low scores on Logical reasoning (T67 below 21 at 6.44)

whereas the key IPC influencers include a:

  • Low degree of Card movements (T00 below 22 at 4.54)
  • Inadequate Exploration (T05 below 28 at 1.77)

The Administrators generally show under-developed cognitive skills, particularly where it comes to Exploration processes which are crucial to identify the relevant information which more complex processing relies on. Analytical skills training with administrators showed them responsive to the repeated practice and internalization of the metacognitive criteria aimed at optimizing Exploration skills, including: “Is this important?”, “Is this clear?” and “Do I need more information?”. Their cognitive effectiveness also tends to improve by developing the habit of making lists and representing information in simple structures or graphs.

 

Engineers

In the case of older Engineers, the stylistic differentiators were:

  • Explorative (T61 above 26 at 8.55)
  • Integrative (T70 above 46 at 2.25)
  • Holistic (T64 above 45 at 2.17)

The IPC influencers were:

  • Exploration (T05 above 24 at infinity)
  • Detail (T17 above 75 at 4.48)
  • Precision (T11 above 45 at 2.60)

It thus seems that older Engineers, more so than most other career groups except IT developers, invest the bulk of their energy in the detailed exploration of unfamiliar situations. Their Exploration processes (as measured by “explorative tendencies” or T05 as opposed to T43 which measures “effective exploration processes” only), may not necessarily be economical or carefully directed. They therefore tend to look for information from various perspectives and may over-explore at times. While exploring as widely and in such depth as they tend to, which reflects a detail orientation, they also manage to continually integrate relevant information to understand the situation and formulate a solution.

Compared to other career groups, younger Engineers also showed a rigorous detailed and integrative approach as indicated by their preferred cognitive styles of:

  • Analytical (T62 above 64 at 2.66)
  • Logical (T67 above 61 at 2.50)
  • Integrative (T70 above 63 at 2.46)

In terms of their IPCs, the younger Engineers could be recognized by their higher scores for:

  • Metacognitive strategies for Logical Reasoning (T30 above 69 at 2.90)
  • Metacognitive Exploration Strategies (T06 above 69 at 2.76)
  • Analytical Processes (T40 above 58 at 2.43)

Given their strong metacognitive awareness and the degree of self-monitoring of their thinking processes, young Engineers thus deal effectively and economically with new and unfamiliar information. Both the older and younger engineers do, however, show a technical and rigorous approach which to some extent differentiates them from other career groups. Action research through personal feedback, aimed at identifying the cognitive characteristics of Engineers, has indicated that this career group, more so than most others, are cognitively diverse and rely on a wide range of cognitive preferences and capabilities.

 

IT Developers, Programmers and Data Analysts

This group seem best differentiated by their cognitive style preferences for:

  • Learning (T72 above 46 at 2.09)
  • Reflective (T74 above 50 at 1.99)
  • Logical (T67 above 50 at 1.94)

Their most characteristic IPCs included:

  • Exploration (T05 above 29 at 4.82)
  • Low Coherence (T16 below 17 at 4.27)
  • Precision (T11 above 45 at 2.32).

Here the averaging effects of group diversity somewhat obscured the strong visual mode of processing and well-developed learning and analytical orientation of programmers as found through qualitative and action research investigations. The low Coherence scores, which is measured by the CPP according to various aspects of conceptualization, using auditive modes of processing, do however reflect the visual orientation of this group which has repeatedly been confirmed in case studies and personal feedback.

Action research has also indicated IT developers and programmers as the most Learning oriented of all the career groups. This group generally consists of younger people who work in fast-changing environments which tends to attract those who are cognitively adaptable and agile. Qualitative action research also indicated that besides Quick insight Learning (T15), this groups is also characterized by an Intuitive (T65 and T54) approach and their personality characteristics include openness, curiosity, a need for challenge and self-confidence regarding their intellectual prowess.

 

Lawyers and Attorneys

Those from the Legal fraternity can be expected to differ from other groups’ stylistic preferences in terms of:

  • Random (T69 below 63 at 1.48)
  • Metaphorically inclined (T75 above 65 at 1.47).

In terms of their IPCs the key influencers were:

  • A Focus on External Clues (T1 above 22 at 11.60)
  • Self-Monitoring (T36 below 66 at 2.05)
  • Task Orientation (T35 between 32 and 63 at 1.97)

Based on action research involving personal feedback to people from this career category, they generally seem to show complexity preferences for Tactical Strategy work – in some cases with “potential” for Parallel Processing work complexity. They also, almost invariably, showed the fairly unique combination of a factual, Logical with an ideas-oriented, Metaphoric cognitive styles. Other than alternative career groups such as the Accountants, Programmers and Engineers who are analytically inclined, Lawyers thus seem to be more interested in the world of ideas, they often capitalize on metaphors and analogies when explaining concepts, tend to use abstract language and apply a metacognitively rigorous approach to problem solving. The latter findings based on informal action research are thus partly verified by the AI results on this sample of legal experts.

 

The Social Sciences Career Group

For this group which includes HR practitioners, Social workers and Teachers the processing style differentiators revolved around average to above average:

  • Random (T68, 50 – 71 at 1.48)
  • Low Integrative (T70 below 50 at 1.47)
  • Low Logical (T67 below 50 at 1.47)
  • Low Reflective (T74 below 50 at 1.43)

The best IPC differentiators seemed that of:

  • Speed (T56 below 70 at 2.53)
  • Low Metacognitive monitoring of Analytical Processes (T12 below 52 at 1.74)

Compared to most other professionally trained career groups, those from Social Sciences seem somewhat less effective in terms of their cognitive competence in that this group shows relatively low metacognitive awareness, as well as low integrative and logical skill. Action research as well as quantitative research amongst social sciences groups have indicated a widely diverse group, though. In terms of the levels of complexity these individuals seem to display capabilities ranging from Pure Operational to Pure Strategic SST Levels. Their stylistic preferences and information processing competencies also widely differed. It would be misleading to generalize specific processing findings for this group.

 

Marketing and Sales

In the case of these career groups the sample showed the key cognitive style influencers to be that of:

  • Memory (T66 below 73 at 3.89)
  • Holistic (T64 above 38 at 1.59)
  • Metaphoric (T75 above 35 at 1.57)

IPC influencers were those of:

  • Low on the Degree of Detail Focused on (T17 below 20 at 6.40)
  • High Memory (T31 below 74 at 3.19).

This particular combination means that although this career group relies on detail, this detail is mostly based on previously acquired knowledge and experience (as opposed to the tendency to pull situations apart in order to understand the building blocks and their interrelationships) as opposed to fresh analyses of unfamiliar information. Within an unfamiliar problem-solving situation, they may thus fall back on a more general problem-solving approach.

Again, action research on the cognitive characteristics of people in media, journalism, marketing and advertising showed a clear ideas orientation and tendency to capitalize on verbal conceptualization (as reflected by the Metaphoric style tendency). Those involved in the formulation of creative advertising strategies, often seem to intentionally apply a particularly Random and Impulsive approach. More characteristic of this Marketing and Sales career group than their cognition, however, is their worldview as indicated by the Value Orientations (VO) assessment.

On the VO, those in Marketing roles often show an “Orange” orientation characterized by a focus on the creation of value; skill in manipulating perceptions; a belief in their capability to create their own future; personal resilience; as well as a creative strategic orientation. This worldview has been found to prevail across regions and is as prevalent in Africa as it is in Europe, Asia or the Americas. Within Sales the Red value system as measured by the VO was the most common preferences. The latter is associated with high energy as well as a degree of forcefulness and a need for achievement and proving oneself.

 

Medical Doctors

This career group was best differentiated from other career groups based on the following stylistic and processing characteristics. In terms of cognitive style:

  • Reflective (T74 above 43 at 1.68)
  • Low Impulsivity (T69 below 55 at 1.64)
  • Analytical (T62 above 44 at 1.62)
  • Quick Insight (T73 above 42 at 1.60)

In terms of their IPCs, differentiators included:

  • Spontaneous comparison (T09 above 33 at 2.20)
  • Strategizing (T37 above 42 at 1.78)
  • General approach (T04 above 45 at 1.74)
  • Detail (T17 above 44 at 1.70).

A general observation is that those in healthcare and medicine are often academically astute.  They also tend to capitalize on knowledge and experience, tend to be factual, realistic and practically inclined and, probably given the risks involved, are relatively conservative in their approach – therefore the tendency to spontaneously compare elements and to be Reflective as opposed to Impulsive regardless of their characteristic Quick Insight. The majority show awareness of the integrated nature of their subject matter by apply systems thinking. However, action research results suggest that over time, career-related fatigue often seems to result in a more linear and structured approach in solving problems, as opposed to the systemic view required for medical careers.

 

Politicians

Caution should be taken to generalize the results of this group of Politicians as the sample largely consisted of managers, director generals and others in executive roles in the public sector in South Africa. Key stylistic differentiators included:

  • Low Effectiveness in Explorative (T61 below 27 at 3.77)
  • Low Logical Reasoning (T67 below 31 at 3.50)
  • Low Reflective (T74 below 33 at 2.94)
  • High Random (T69 above 66 at 2.81).

Key IPC differentiators included:

  • Few and ineffective Card Movements (T00 below 27 at 4.30)
  • Inadequate Strategies to deal with Complexity (T25 below 28 at 4.20)
  • Low tendency to look for Logical Evidence (T29 below 29 at 3.66)
  • Undeveloped Analytical Skills (T40 below 32 at 3.16).

In terms of their results regarding cognitive complexity and ideal SST environment, the majority of these individuals showed Pure Operational and Diagnostic current levels of SST work complexity and a small proportion showed potential for Tactical Strategy work.

 

Conclusions

In constructing an evidence base for the CPP, which transcends the shortcomings of conventional Psychometrics, while also pursuing the further understanding of cognitive competence across groups, the contribution of qualitative and quantitative research techniques was also evaluated.

Based on approximately 50 formal empirical studies aimed at the validation of the CPP (as reported on in the CPP Technical Manual); countless smaller empirical  case studies aimed at quantifying and clarifying group-specific cognitive characteristics; qualitative research conducted via personal feedback to approximately 5,000 individuals where alternative test results were mostly available; and big data AI investigations, it seems that the most viable research approach in dealing with psychological data is one characterized by a focus on smaller, homogeneous and contextualized samples, where subjective insight and experience is capitalized on. Such case studies seem to produce clear and intuitively appealing results – more so than big data or even AI analyses do. It is, however, useful to combine big data AI analyses with a qualitative case-study approach to best understand the complexity and dynamics involved in cognitive functioning.

From the studies reported on in these blogs, it seems that the most effective and complex thinkers across career groups apply relatively similar cognitive approaches in that they capitalize on thinking skills “high up” in the holonic structure of the information processing model (IPM) – processes which are of an encompassing and inclusive nature. High level performers across career groups seem metacognitively aware and generally apply a Logical, Integrative and Learning orientation while capitalizing on the full spectrum of cognitive competencies.

However, it is also of interest to differentiate between the cognitive characteristics of various career groups. These differentiators are obscured by normal statistical practice which is subject to averaging effects. Artificial Intelligence (AI) techniques come to the rescue in this regard in that it reveals typical career group-specific cognitive indicators as discussed above. The AI findings seem to verify conclusions based on qualitative action research using a case study approach.

 

A summary of the findings

Cognitive characteristics of top performers on the CPP:

Styles:

  • Integrative (T70)
  • Logical (T67)
  • Holistic (T64)
  • Learning (T72)

Information Processing competencies (IPCs):

  • Learning (T44, T15)
  • Metacognition (T37)

 

Table 4 summarizes all the key features associated with the career groups analyzed in the three blogs constituting this series.

Table 4: Career specific cognitive competencies

CPP Career Group Analytics: High performance cognitive profiles across career groups

By Christoff Prinsloo and Maretha Prinsloo on December 23, 2019

© 腾龙 郭/ adobe.stock.com

 

Introduction

This study involved a number of investigations regarding the stylistic and information processing preferences and capabilities of candidates from various career groups who showed high levels of performance on the CPP. Normalized-Standardized T-scores (NST-scores) were used here, with ranked scores used for the analysis of the styles, and the scores themselves used for information processing preferences.

 

Analytics Sample Information

Again, somewhat homogenous groups were selected in terms of age, educational level and educational field. From these samples, only the top 10% in terms of CPP performance were selected. The goal was to determine whether qualitative versus quantitative aspects of information processing prevail across career groups. Significant differences between top cognitive performers across groups may indicate that educational and work experience largely shape a person’s cognitive approach to problem solving. A lack of differentiation between groups may indicate that top cognitive performers on the CPP, across the career spectrum, capitalize on similar cognitive competencies. The group information is provided in Table 1.

Table 1: The career group samples selected

 

The results below reflect the top 10% in terms of cognitive capability on the CPP. The analyses were done in terms of both cognitive style as well as information processing competencies (IPCs) as measured by the CPP. Ranked Normalised-Standardised T-scores were used. An analysis of the cognitive style preferences of the top 10% of candidates, indicated almost no significant differences between the stylistic preferences and small differences in terms of information processing preferences of various career groups.

 

Cognitive Styles

Two examples of the many tree structure representations generated, depicting the stylistic preferences of Lawyers and Accountants (older), appear below in Figure 1 and Figure 2.

Figure 1: Tree structure representation of the cognitive style preferences of the top 10% of Lawyers

 

Figure 2: Tree structure representation of the cognitive style preferences of the top 10% of Accountants (older)

 

The results of the tree structure analyses of the Cognitive Style and Complexity preferences of the top 10% CPP performers from the various career groups can be interpreted in the following way:

  • Almost all those in the top 10% (on CPP performance) achieved a current Tactical Strategy SST level of work orientation or higher, which means that in terms of the unit of information utilized, they preferred to work with tangible systems (Tactical Strategy level) and/or dynamic systems (Parallel Processing Level) and some even showed the capability to deal with chaos and emerging patterns (Pure Strategic level) at times. The unit of information that they preferred to work with thus indicates a tendency to deal with complex information as opposed to merely considering separate elements or linear causality.
  • Although there were some fluctuations between career groups in terms of the order of their Style preferences, the most prominent stylistic preference of the top 10% across career groups was the Integrative style. It constitutes a highly effective cognitive approach aimed at making meaning of new and discrepant information. The Integrative style involves the tendency to deal with complex, vague and dynamic information through the continuous synthesis of elements as they are encountered. Those showing an Integrative approach to information processing thus tend to explore new information according to hypotheses and metacognitively directed strategies.

Further interesting findings regarding the use of the Integrative style include:

  • An integrative approach to understanding one’s world is often characterized by an ideas-orientation; well-developed techniques to differentiate between relevant and irrelevant information; as well as continuous ‘meaning making’ and thus a focus on the management of the task’s complexity.
  • Integrative thinkers tend to discard irrelevant information relatively early in the process of exploration and show a strong tendency to reinterpret and link new information to existing conceptualizations – almost as is described by Piaget’s concepts of assimilation, accommodation and equilibration.
  • The use of an Integrative style is typically and critically required in theoretical or academic research environments where discrepant information is synthesized into a meaningful and coherent whole or representation. Integration is thus a prerequisite for cognitive functioning at a Parallel Processing level of work complexity which requires model building given vague, interactive and dynamic aspects.
  • The Integrative style is characterized by a reliance on hypothesizing and the continuous verification or falsification of hypotheses to manage the task complexity. From findings in the literature it seems that only 15% of graduates actually falsify hypotheses in everyday problem solving. The rest merely tend to verify their hypotheses or jump to conclusions to avoid the uncertainty and discomfort of cognitive dissonance. Only a small percentage of graduates therefore develop an Integrative approach to problem solving.
  • The only two career groups that did not capitalize on an Integrative cognitive style as a first preference, are the Administrators and those from the Sales and Marketing category. The top 10% of candidates in terms of their CPP scores in these two career groups, applied a rigorous Logical approach to problem solving.
  • The second most commonly applied stylistic preference of the top 10% of CPP performers across most of the career groups, was that of a Logical reasoning style. Those showing a logical approach to problem solving tend to deal with analytical facts, but given their need for cognitive challenge, prefer to utilize these facts to build arguments. They thus tend to apply a rigorous, process-oriented approach and follow their arguments through to identify conclusions, implications and consequences – thereby contextualizing their conclusions. They are therefore seldom satisfied dealing with the facts only but tend to apply either convergent or divergent processes to build on the facts and take them a step further. During this process they continually look for logical evidence in a rigorous manner and are unlikely to revert to mere assumptions.
  • Although most of the top CPP performers of the various career groups showed a Logical style as a second preference and capability, the older Accountants showed a slight preference for the application of a Holistic as opposed to a Logical approach as second preference. This big picture tendency seems typical of older professionals across the board. On average, older Accountants (46+ years of age) do, however, also apply a Logical style as their third preference.
  • The third preference of the top 10% of CPP performers across career groups is that of the Holistic style. This can be described as a preference to look at the big picture without necessarily neglecting relevant details. In practice it differs from the Integrative approach in that an Integrative style capitalizes on discrepant, vague and complex information to build a new theory, model or concept, whereas the Holistic style is characterized by a generic or big picture approach and the identification of, and focus on, the core elements involved with the aim of their practical application, as is often the case in business. In addition, abstract and general language is frequently used and there is a slight resistance to descend into technicalities unless the application of a detailed element or consideration is likely to affect the big picture. The Holistic style is therefore both an ideas-driven and practical approach that can be implemented in practice due to its emphasis on core elements.
  • The fourth stylistic preference of the top 10% CPP performers across career groups is that of Learning. This approach is characterized by interest, energy, curiosity, attention, metacognitive awareness, flexibility and openness as well as the use of memory. Although the use of a Learning style is generally more common amongst younger rather than older people it therefore also seems to characterize those showing highly effective cognitive approaches regardless of their age or field of interest. This stylistic preference is becoming increasingly important within fast changing work environments where a person’s knowledge base and previous experience are not as important as their adaptability, metacognitive awareness and motivational drive to master new situations.
  • Surprisingly, the only career groups that did not show Learning as a fourth stylistic preference, were young Accountants (24 – 44 years of age) who showed a more Intuitive inclination (although the Intuitive and Learning styles do overlap); older Engineers who reverted to a highly Analytical Approach; and the top cognitively performing 10% Politicians who showed a Reflective style as a fourth preference – a cognitive style which none of the top level CPP performers in any of the other career fields showed.

 

Information Processing Competencies

Not only the cognitive styles of various career groups were analyzed but also the ranked NST-scores of the Information Processing Competencies (IPCs) of the top 10% CPP performers across all the career groups. Figures 3 and 4 indicate the results of Administrators and younger Accountants in this regard.

Figure 3: The information processing competencies most capitalized on by the top 10% of Administrators

 

Figure 4: The information processing competencies most capitalized on by the top 10% of Accountants (younger)

 

In terms of the ranked Information Processing Competencies (IPC’s) which characterized the top 10% of CPP performers across the various career groups, the following findings emerged:

  • Almost all the top CPP performers across groups except Politics, seemed to achieve the highest scores on IPCs indicating Learning (T44 and T15) and Metacognition (T37), thereby indicating their cognitive modifiability and flexibility as well as their metacognitive strategic awareness.
  • The top CPP performers across a number of career groups showed a marked Detail approach (T11, T17). Included are those in Medicine, Law, Finance (other), Administration, Politics as well as younger and older Engineers.
  • Of these, Lawyers, Medical doctors, Marketers, older Engineers and Politicians seem to capitalize equally on detailed (T11, T17) as well as Systemic / big picture (T18, T19) processes. In the case of young Engineers, their detailed approach (T43, T11) was combined with a Learning orientation (T44).
  • The top CPP performers in the Political career group, however, showed a strong tendency to meaningfully and holistically interpret information (T18, T19, T64).
  • Finance (other) which included Bankers, Economists and others in the financial field, was the only group characterized by the tendency to make Assumptions (T46), regardless of the fact that this entire sample formed part of the top 10% of CPP performers. It may indicate habitual carelessness and the tendency to apply a commonsense approach or to rely on their previous experience regardless of their cognitive capability.
  • Whereas the older Accountants showed a higher likelihood of relying on Systemic / Big picture (T18), Generic (T04) and Metacognitively strategic (T37) processing, the younger Accountants capitalized most on Learning (T44) and Memory strategies (T32).
  • Younger Accountants depended on their well-developed Quick Insight Learning orientation (T15).
  • The Actuarial career group applied attentive, Metacognitively directed (T37, T43), Rule-based (T10), Logical (T55) and Learning (T44) orientations.
  • Older Engineers did not rely as much on Learning as on Integrative (T19) and Metacognitively aware processing (T37).
  • Older Engineers and older Accountants, more so than their younger counterparts, also relied on Metacognitive alertness (T37) and on Integrative / Systemic (T18, T19) processes. This combination may indicate wisdom and intuition with regards to their fields of specialization.
  • Younger Engineers and younger Accountants both showed a stronger Learning orientation (T15, T44) than their older counterparts.
  • Of all the career groups, the young Accountants depended most on going about in an Economical way (T33) in dealing with unfamiliar information. They were also the one group that relied most on Memory Strategies (T32).

Again (in using ranked NST-scores) it can be concluded that the top 10% of CPP performers across career groups show striking similarities in terms of the cognitive processes capitalized upon.

The results of this study again verify the finding that effective thinkers utilize the full spectrum of thinking processes in a metacognitively directed and flexible manner and that the impact of cognitive competence (the quantitative aspect) to some extent overshadows the impact of specific career-related processing practices (the qualitative aspect) in determining cognitive preferences and habits.

 

Age, Educational Attainment, and SST Levels

In this additional tree structure study, groups were selected based on the following criteria: varying age, SST level, and Educational level aimed at effectively supporting cognitive functioning per career category. Normalized-Standardized T-scores (NST-scores) were again used here, with ranked scores used for the analysis of the styles, and the scores themselves used for information processing preferences.  The following findings emerged:

Table 2: The career group samples selected

 

Summarizing the results of this analysis:

  • As for the analyses of the groups outlined in Table 1, most of the high-level functioning individuals across the career groups of this sample again showed a preference for the application of an Integrative cognitive style which was closely followed up by Logical, Holistic and Learning approaches.
  • The Actuarial, Data processing as well as Marketing and Sales groups also capitalized on Structuring stylistic tendencies.
  • The Medical, HR, Data analysis and Research groups showed Intuitive and Quick insight style tendencies too – probably due to the many possibilities in interpreting the complex and vague information they deal with in their career fields.
  • The Actuarial group to some extent differed from all the other career groups in that they showed almost equally high scores on most of the processes measured by the CPP – indicative of both operational and strategic capability. None of the other groups did in fact show as a pronounced tendency to likewise deal with both detail and big picture information; to structure and categorize information on the one hand, while also taking a systemic or big picture perspective; to focus on facts as well as processes; to accommodate both tangible and intangible information, etc.
  • In this sample, Finance (non-accounting), Administration and Legal groups seemed to capitalize on Memory, or previous acquired knowledge and experience.

Although the results of the data from the various groups described in Tables 1 and 2 closely resemble one another, it seems that the selection of the samples is critical in identifying career specific processing tendencies.

CPP Career Group Analytics: Career related processing preferences

By Christoff Prinsloo and Maretha Prinsloo on December 18, 2019

© 腾龙 郭/ adobe.stock.com

 

Introduction

Cognadev’s CPP database which currently consists of approximately 400 000 sets of cognitive results, offers interesting insights into the intellectual functioning of various groups within the work environment, globally. The results span the cognitive preferences and complexity capabilities of candidates including their cognitive styles of thinking, information processing competencies (IPCs), units of information used and their learning potential. As such the insights gained from the data underline tendencies in cognitive functioning that are generally expected from certain age, ethnic, gender, educational level, educational field, employment category and other groupings. New insights are also offered with regards to the cognitive approaches of various career groups across regions. In this document, examples of the findings pivoting around a few interesting areas are reported.

A number of investigations on cognition have, over the years, focused on whether some commonly held beliefs about organisational roles are supported by CPP results. The aim therefore was to test whether the strengths and weaknesses traditionally associated with specific career paths held true, and whether new insights could be gained. This three-part blog series reports the results arising from an investigation of cognitive differences between various career categories such as engineers, marketers, accountants and administrative personnel, by means of three visualisation/analytics approaches:

  • In part 1 of this blog series, the processing preferences of various career groups were visualised using two-dimensional scatter plots to investigate common expectations of their cognitive functioning.
  • In part 2, the cognitive styles and information processing tendencies of top CPP performers across various career categories were analysed and visualised by means of tree structure representations to investigate the impact of career specific processing requirements and practice on cognitive functioning versus the impact of general cognitive competence on cognitive functioning.
  • In part 3, we report the results of an application of an Artificial Intelligence technique used to identify the cognitive style and information processing competencies as measured by the CPP, which best differentiate between various career groups. The findings were compared to results emerging from case study-based action research using empirical or quantitative as well as qualitative investigations.

 

Analytics Sample Information

For the purposes of this study, the following sample was selected as a target group for exploratory investigation.

  • Level-of-work: Only SST Level 3, alternatively referred to as Tactical Strategy (TS) or Alternative Paths (AP) level of work
  • Age at assessment: 35-54
  • Level of Education: Multiple Degrees

This yielded a sample size of 13,100 candidates, spread across a multitude of functional areas, disciplines of qualifications and sectoral involvement. Some examples of the many analyses that were conducted, are shown here.

 

Functional Area Analytics

Using the CPP-based Standardised T-scores, the aim of the visualisations below was to determine whether different functional areas consistently exhibited different information processing competency (IPC) profiles and differently ranked cognitive style preferences; and to what extent they mirrored expected cognitive characteristics. Results based on selected combinations of 56 IPC and 14 style scores were used. An example of the Metaphoric versus Analytical style scatter plot for various functional areas is shown below to illustrate cognitive processing characteristics of the various groups. The results represented in Figure 1 mirror our core understanding of typical job processing characteristics, with Engineering/Technical preferring an Analytical Thinking Style. Likewise, it is no surprise to see Customer Service and Teachers/Lecturers/Social scientists groups opting for a Metaphorical, verbally oriented, approach.

Figure 1: Functional groups: Metaphoric versus Analytical styles

 

The second example of the visualisations focusing on Employment Sectors depicts a two-dimensional graph or scatter plot regarding their information processing competency (IPC) scores.  The results shown in Figure 2 exhibits a plot of Judgement versus Analysis, showing IT dominating Judgement scores and again Legal, Pharmaceutical and Mining presenting top Analytical scores. The Retail and Food and Beverage as well as Consulting industries achieved lower average scores in terms of these two processing scores.

Figure 2: Sector of employment: Information processing competencies – Analytical versus Judgement IPC scores

 

Discipline of Education Analytics

The next sub-sample came in lieu of discipline of qualification. So, although an employee might have ended up in a managerial functional area, and in Retail, he/she might have had a scientific undergraduate qualification. This analysis was therefore done to investigate whether an employee’s fundamental academic training shaped their thinking, as opposed to the jobs they now fill.

Figure 3 shows an example of the scatter plot visualisations done. It again contrasts the Analytical and Metaphorical thinking styles, using discipline of qualification as the core dimension. The results indicate the clear Metaphorical thinking style for those from Marketing and Sales disciplines, while Military trained groups showed a clear Analytical thinking style preference. The long list of similar good analytical performers are the usual suspect of science and engineering. These results seem intuitively appealing.

Figure 3: Discipline of qualification: Metaphoric versus Analytical styles

 

Our final graph depicted here in Figure 4 shows Judgement versus Story telling Information Processing Competencies (IPCs). Not surprisingly, Marketers have found their niche, while Military personnel still know how make the tough calls objectively whilst the rest seem to cluster together and therefore show equally developed skills in this regard.

Figure 4: Discipline of qualification – Information processing skills of Judgement versus Story telling

 

The above findings to some extent exhibit the depth of insights that can be gained from Cognadev’s CPP database through the use of a reputable business intelligence platform. A number of interesting insights were revealed by the entire exercise. Most of these confirmed our understanding of the job market as it operates now, but the many different analyses also revealed a few surprises.

Increasing age and changes in CPP preferred cognitive style across job families

By Paul Barrett and Maretha Prinsloo on December 13, 2019

© Sergejson / adobe.stock.com

 

“… one cannot really tell if a successful person has skills, or if a person with skills will succeed – but we can pretty much predict the negative, that a person devoid of skills, will eventually fail.” (Nassim Taleb, Antifragility, p 303)

Here, we follow up on the previous blog/article on cognitive stylistic preference versus cognitive power to explore Taleb’s statement which represents a subtractive epistemology. This is done by analysing the cognitive styles of various age and career groups to determine which cognitive styles best differentiate between groups and whether we should rather avoid psychometric profiles which could potentially hold risk. Or as Taleb puts it: “just work on removing the pebble in your shoe” as knowledge grows by subtraction much more so than by addition.

Could a simple method of forecasting in psychometrics work better than complex approaches and how could this contribute to our understanding of people which is a prerequisite for effective talent management?

In this investigation, we looked at how preferred cognitive styles vary over different job families and age-groups, equated on their education (possessing a single degree). A sample of the most recently acquired 60,572 cases of CPP data were used, subdivided into four age groups (20-29, 30-39, 40-49, 50 and above) and only those cases who possessed a single degree qualification. We computed the median ranked style for each of the 14 CPP cognitive styles, within each age-group, across 10 job families.

To represent the graphs of the average stylistic preferences of all the various age and career groups here may be counter-productive. Interesting differences between groups may be obscured by the equilibrium forces characteristic of statistical methods in psychometrics. Given the fact that the research aims and the subject matter remain key in applying a most appropriate research method, the cognitive styles of specific groups will be analysed here to get an idea of possible outliers which may differentiate between groups.

The most prudent way of encapsulating the stylistic differences between groups is to identify substantively varying medians across both job families and age groups. The five ranked styles showing most changes in median preferred ranked style across ages highlighted by the shaded areas:
Random, Metaphoric, Explorative, Memory, and Impulsive.

We can now look in detail at each of these substantively varying ranked styles.

Figure 1: The Random preferred cognitive style rank across job family and age group

The trend here is that for all job families except Technical-Engineering-Research and Manufacturing-Construction, as individuals age, so do they show a preference for a less systematic/rigorous approach to working with information rather than systematically analysing, structuring or reasoning about issues.

Figure 2: The Metaphoric preferred cognitive style rank across job family and age group

There is less variability with age for this style, except for the older group of Teachers/Trainees who show an increasing preference for capitalising on auditive modes of processing and viewing cognitive challenges from abstract, creative and/or symbolic angles. As individuals in this particular job family age, it seems their preference for conveying information and aligning the perceptions of others is increasingly achieved by adopting the use of powerful metaphors. However, we must be cautious in our interpretation here as the sample size for this specific job family 50-and-above age group is just 17 cases.

Figure 3: The Explorative preferred cognitive style rank across job family and age group

An explorative style is preferred by someone who thoroughly investigates different types of information but may get confused by over-exploring and checking too much, resulting in repeatedly revisiting the same information without moving forward. Again, we have to be cautious here in over-interpreting the 50-and-above age group data as sample sizes for Manufacturing-Construction and Creative-Media groups are 18, and 11 respectively. However, for the Technical-Engineering-Research group, the sample size is 76. On balance, there are few systematic age-related general trends for this style.

Interesting hypotheses can, however, be inferred here, for example that older individuals in creative and technical career fields may rely more on previous experience and personal insights than on exploring unfamiliar external sources of information. The 20 – 29 year old accountants, of whom many are trainees or interns, may not explore additional and unfamiliar sources of information as widely as their older counterparts in accounting do. The most explorative in the 50 – 59 year age group seem to be those in accounting, marketing and teaching; in the 40 – 49 year age group, those in manufacturing; in the 30 – 39 year age group those in creative and marketing careers; and the 20 – 29 year age group those in human resources, teaching and marketing. For most age groups the more creative career fields are thus associated with an explorative approach whereas for those in the construction or manufacturing fields, characterised by practical risks, exploration is as important.

Figure 4: The Memory preferred cognitive style rank across job family and age group

The two most obvious changes in style preference with age are within the Human Resources group (n=75) and Creative-Media group (n=11), with the aged 50 and above group showing an increased preference for using memory strategies to process information/formulate solutions. A preference for a memory style of working within an individual is exemplified by a reliance on past experience and a knowledge base, internalisation and integration of information while processing it, and a tendency to use memory strategies such as confirmation of hypotheses, external reminders, visualisations and associations. As we grow older, experience is embedded in our memories, and we tend to employ more of that stored information resource in our decision-making.

Figure 5: The Impulsive/Reactive preferred cognitive style rank across job family and age group

Similar in some respects to the data in Figure 3, for the Random style. Older age groups within certain job families show an increasing preference for an impulsive or reactive cognitive style. An individual showing a preference for this style of cognition may respond to problems emotionally rather than rationally, favouring quick solutions over taking longer but being more accurate. An element of impatience is associated with this stylistic preference.

Others may observe that the impulsive individual may not spend sufficient time on complex cognitive challenges, preferring instead to make quick decisions under conditions of uncertainty. The two job families where the least median preference is shown over all ages are the Technical-Engineering-Research and Manufacturing-Construction groups. This tendency may be related to the risk associated with judgement errors in these career fields.

In Conclusion

The five cognitive styles which best differentiate between the age and career group are the Explorative, Memory, Metaphoric, Random / Trial-and-error, Impulsive / Reactive approaches.

In addition, the CPP ranked preferred cognitive styles show some interesting and intuitively appealing trends across age and job families. The ‘standout’ finding is that a preference for a more undirected action, reliance on previous experience (memory), and impulsive (quick closure) approach to decision-making is more prevalent among older-aged groups within job families which do not incorporate a substantive technical component (such as Engineering and Manufacturing).

This further substantiates the findings of a number of previous studies on both strategic effectiveness and on the predictive validity of the CPP, where these cognitive styles have indicated a somewhat operational and less effective information processing approach than that associated with the Logical, Integrative, Holistic, Learning and Quick Insight styles.

If we heed Taleb’s advice to predict performance in terms of negative as opposed to positive indicators, these somewhat less effective cognitive styles, which also best differentiate between age and career groups, may also best differentiate between individuals within groups, and may flag potential risk for HR decision on selection, placement and development.

Cognadev Technical Report #11 provides the detailed sample information and descriptive statistics from which this blog article has been compiled.

Cognitive complexity and cognitive styles: implications for strategic work

By Paul Barrett and Maretha Prinsloo on December 13, 2019

© Yulia / adobe.stock.com

 

This research study investigates the nature of the proposed holonic model of information processing constructs on which the Cognitive Process Profile (CPP) is based and its implications for strategic work. This is done by determining the relationship between high versus low levels of information processing competence (as a measure of cognitive complexity), and cognitive styles (as a measure of cognitive preferences associated with the inherent altitude and inclusivity of the proposed holonic model).

The Cognitive Process Profile (CPP) assessment

The CPP measures a person’s cognitive preferences and capabilities by means of a simulation exercise which was designed to externalise and track thinking processes according to thousands of measurement points. The results are analysed by an expert system and automated reports are generated. The CPP primarily measures the following constructs: information processing competencies, cognitive styles, units of information, learning potential, a suitable working environment, as well as cognitive strengths and development areas.

The theoretical model of thinking processes

The CPP is based on a self-contained theoretical model of thinking processes. The Cognadev Information Processing model (IPM) is holonically organised in that the various thinking processes are represented as a “soft hierarchy” of increasingly complex and inclusive operations. A holon refers to a system which consists of various subsystems, each of which incorporates and transcends underlying subsystems. The thinking processes incorporated in the CPP model can be regarded as functional information processing categories.

Information processing constructs

Each of these information processing constructs consists of a number of sub-constructs, all of which are guided by the application of metacognitive criteria. For example, the processing construct of Exploration consists of sub-processes including scanning, searching, focusing, hypothesising, investigating, discriminating, selecting and eliminating information. The metacognitive criteria that guide exploration activities are those of clarity, relevance and depth. A high score on the information processing construct of Exploration, indicates the effectiveness by which a person investigates unfamiliar information.

Not all the information processing constructs measured, indicate effectiveness in thinking, though. Some processes such as Quick closure and Assumptions amongst others, may actually indicate ineffective thinking strategies or an absence of metacognitive awareness.

The theoretical model of thinking processes on which the CPP is based, can be depicted graphically as follows:

The various metacognitive criteria responsible for the effective application of each of the processes are shown below:

Cognitive Styles

The cognitive styles as measured by the CPP primarily describe the cognitive preferences a person shows in dealing with unfamiliar information. However, it is highly likely that the person will generally apply those same stylistic preferences in familiar contexts also. Cognitive styles can be described as broad cognitive response tendencies and should be understood as the most frequent behaviour during the assessment.

The definition of the particular styles may not be exactly what is generally in layman’s terms associated with the title word. Logical style, for example, implies disciplined thinking in a consequential and process-based manner to transform information structures or to identify implications and consequences. This goes beyond the meaning of the layman’s term “logical”.

A person’s stylistic preferences can be magnified by certain personality and environmental factors as well as value orientations. An example is the Reflective style, which may indicate a level of caution, a risk avoidant personality trait, internalised cultural values or possible exposure to high risk or punitive environments where mistakes are not tolerated. Certain stylistic tendencies are also reinforced or adopted in certain educational and work environments. Examples include the highly analytical requirements of certain financial and scientific career fields, or the creative, intuitive, at times even random, ideas-oriented approaches required by arts and, to some extent, the social sciences. Preferences for the application of particular styles, can thus be rooted in cognitive “values” or habitually applied metacognitive criteria. Included are the tendencies to strive for accuracy; the habit to suppress reactive responses in favour of being reflective; or the tendency to create certainty by approaching tasks in an ordered and structured manner.

Other than the information processing constructs, such as Exploration, which indicates effectiveness of processing, cognitive styles such as the Explorative Style, may merely indicate the tendency to explore irrespective of the effectiveness involved. Typical cognitive styles which fall into this category include the Explorative, Structured, Reflective, Random or Trail-and-Error and Memory styles. The Intuitive and Analytical styles can partly be grouped into this category as well.

However, the Logical, Integrative, Holistic and Learning styles, presuppose effectiveness of approach. These styles also involve dealing with complexity in an “inclusive” and metacognitively aware manner. Again, the Analytical and Intuitive styles can to some extent be added into this category.

To some extent the various stylistic preferences echo the holonic nature of the proposed information processing model.

The CPP indicates a test candidate’s preference for 14 different cognitive styles by assigning a ranking to them according to the least (1) and most (14) preferred or utilised.

Investigating the nature of the holonic structure of thinking processes

The holonic nature of the proposed processing model reflects a progression towards greater complexity, inclusiveness and metacognitive awareness at higher levels of processing. In terms of the altitude inherent to the holonic model, Integrative and Transformational styles (as reflective of integrative and logical reasoning processes) can thus be expected at higher levels of organisation on the holonic model than those of the Explorative and Analytical styles. The more complex styles can therefore be expected to be associated with greater information processing competence.

Evidence for the altitude and inclusivity of the holonic processing model can be investigated by differentiating between individuals with high versus low processing competence and comparing this to stylistic preferences to determine the degree of inclusiveness or altitude of both the proposed processes of the information processing model, as well as the associated styles.

Information processing competencies (IPCs) and Styles

For the purposes of the current analysis, a sample of the most recently acquired 60,572 cases of CPP data were used, selecting two clearly discrete groups from within this dataset according to a simple filter:

  • Low IPCs; contains individuals who score at or below the 25th percentile (the lower quartile) on every one of the 14 IPCs (n = 796).
  • High IPCs; contains individuals who score at or above the 75th percentile (the upper quartile) on every one of the 14 IPCs (n = 581).

The median ranked styles across all 14 styles of the two groups were then compared, where the lowest (1) ranked style is the least preferred, and the highest (14) is the most preferred/utilised.

Comparing median ranked styles in low and high IPC groups (1=least preferred, 14 = most preferred)

High IPC cases show a greater preference for the cognitive styles Learning, Logical, Integrative and Holistic. Less conclusive results were found in the cases of the Analytical, Reflective, Memory, Intuitive and Structured styles.

Low IPC cases show a greater preference for the cognitive styles: Metaphoric, Explorative, Reactive/Impulsive, Trail-and-error/Random.

What can we conclude?

Styles that largely reflect the use of less inclusive processing skills, were associated with lower levels of processing competence. For example, by comparing the Explorative and Analytical style preferences, it seems that the Analytical style is associated with greater complexity and altitude than Explorative style and therefore of a more inclusive nature within the proposed holonic structure of the model.

The Analytical and the Logical styles are closely related as both reflect a detailed, rule-based and rigorous approach to problem solving. In the case of the Analytical style, the emphasis is on subdividing information in a detailed and systematic manner to better understand the interrelationships between elements. In the case of the Logical style the emphasis is on following a rule-based approach through to identify consequences and implications or to restructure, transfer, transform or contextualise a solution. From the results obtained in this study it seems that a preference for a logical style is associated with greater complexity and altitude than that associated with an Analytical style.

In the case of the Structuring, Integrative, Holistic and Metaphoric styles, all are involved in meaningfully conceptualising information. The Integrative and Holistic styles are closely related as some of their building blocks overlap. The two approaches do, however, differ in that the Integrative style is largely associated with an interest in coherence and abstraction, whereas the Holistic style is aimed at considering the big picture while not neglecting detailed elements that may impact on the practical implementation of conceptual solutions in specific contexts. It was found that the Integrative and Holistic styles reflect a greater degree of inclusiveness and altitude than the Structuring and Metaphoric styles.

These findings support the altitude and inclusiveness of processing constructs of the proposed holonic model of processing and indicates the cognitive styles best suited to complex work environments. The implication of this finding is that the Logical, Integrative, Holonic and Learning styles are the best suited to the cognitive requirements of strategic roles in organisations

* Cognadev Technical Report #10 provides the detailed sample information and descriptive statistics from which this blog article has been compiled.

CPP Competency and Style Variation in Younger and Older-Aged Employee Groups

By Paul Barrett and Maretha Prinsloo on December 13, 2019

© Thiago Melo / adobe.stock.com

 

 

Of interest are the differences (if any) shown between younger and older employees, with respect to the magnitudes of Information Processing Competencies (IPCs) and preferred Cognitive Styles, within different employment areas.

In this investigation, we looked at how IPCs and Ranked Cognitive Styles vary over different employment areas and age. A sample of the most recently acquired 60,572 cases of CPP data were used, subdivided into two age groups: between 20-30 and between 40 and 60 years of age, excluding those who were identified as “Trainees”. We computed the median IPC and median ranked style for each of the IPCs and Cognitive Styles within each age-group, across 7 employment areas.

The 14 IPCs as indicated in the CPP report, are expressed as normed T-scores, mostly varying between 20 and 80. The 14 CPP Cognitive Styles are ranked, where 1 indicates the least preferred and 14 the most preferred style.

 

Figure 1 shows the younger and older-aged group medians across 14 IPCs.

 

 

The younger group trends are all solid lines with solid median circles, the older-aged group are the same employment-area colour but, using dashed lines and open median circles.

What we see in Figure 1 is lower IPC medians across all IPCs for older aged employee groups compared to their younger counterparts, with the exception of Categorisation and the Use of Memory.

It is noticeable that the median scores of all the different groups selected here tend to consistently covary across IPCs. This is attributable to the fact that the underlying holonic model of cognition (on which the CPP is designed) assesses processes which are made up of overlapping components which are applied in different sequences / processes. These CPP processes build upon one another and are applied in an integrated way. Certain IPCs primarily reflect capability, or cognitive complexity, whereas other IPC scores largely reflect cognitive preferences related to intellectual habits and emotional needs. For example, the Complexity and Logical reasoning IPC scores, to a large extent reflect current capability whereas the IPCs such as Rule Orientation, Categorisation, Memory, Gradual Improvement Learning and Exploration largely reflect strategies to create cognitive certainty and to be factually correct. For these latter scores, detail and tangible facts are more important that understanding and abstraction. The Quick Insight Learning, Judgement and Integrative IPC scores reflect both capability as well as an emotional need, this time for cognitive challenge and meaning. This clustering of IPC scores is also clearly indicated by factor analyses of the CPP processing scores.

The IPC score distribution curves for the 60,000 sample from which the current groupings were selected are slightly skewed. This is due to the use of Standardised T-scores as opposed to Normalised-Standardised T-scores. Standardised T-scores to some extent reflect the skewness inherent in the distribution of raw scores. For example, the mean for Quick Insight Learning is 50, for Making Assumptions is 47.5 and that for Logical reasoning 54.6. Given this degree of skewness in distribution, medians are reflected in the graphs here, rather than means. Age and employee group effects are nevertheless graphically reflected and interpreted regardless of the impact of the norming and the distribution curves of the IPC scores.

 

Figure 2 shows the younger and older-aged group medians across all 14 Ranked Styles.

 

As in Figure 1, the younger group trends are all solid lines with solid median circles, the older-aged group are the same employment-area colour but, using dashed lines and open median circles. The same colours as those in Figure 1 are used for each employment group.

The medians for each age-group, across employment groups are far more similar overall for the median ranked preferred cognitive styles, except for the Trial-and-error (Random) and Reactive (Impulsive) styles, where many older-aged employment groups show a distinct preference for each in comparison to their younger counterparts. The exception here are the Technical/Engineering/Research and Accounting/Finance groups.

The three older-aged groups showing almost identical medians are: HR, Administration/Operations, and Marketing/Sales/Service. These are also the groups whose median preference for Trial-and-error (Random) and Reactive (Impulsive) styles seem to increase substantially with age.

 

What might we conclude?

Only the most obvious findings are summarised here. Readers should bear in mind that only group averages are reflected here and that within each group, individuals widely vary in terms of their scores.

  1. Most noticeable from the IPC graph is that the younger test candidates (20 – 30 years of age) almost consistently outperformed the older groups (40 – 60 years of age). Only the older Technical/Engineering/Research group achieved similar average scores as the younger candidates. The younger groups also generally applied more effective cognitive styles than the older groups. In fact, the older candidates – the Marketing/Sales/Services group in particular, largely seem to apply Trial-and-error (Random), Reactive (Impulsive), Explorative and Reflective cognitive styles, which are not particularly effective.
  2. The average processing scores of the younger candidates primarily span the SST Diagnostic Accumulation as well as the Tactical Strategy levels whereas the average scores of the older groups largely meet the requirements of the Diagnostic Accumulation level. This is to be expected as the majority of people in the corporate environment (roughly estimated at 80% of employees) show Pure Operational and Diagnostic Accumulation cognitive preferences and capabilities.
  3. A closer look reveals that the younger group obtained higher average scores on all the IPC constructs except that of Categorisation, Memory and Exploration. This may be due to the fact that older candidates capitalise on their previous knowledge and skill, or their well-established cognitive structures.
  4. Of the younger candidates, the Technical/Engineering/Research, Accounting/Finance and the Teaching/Training groups achieved higher average scores than that all the other employment areas. It should be pointed out that the Teaching/Training group here, mostly consists of people in the Accounting field. The younger candidates in the Administrative, HR, Sales, and Creative employment areas generally achieved somewhat lower information competency scores. Here it should be noted that the candidates in the Creative/Media employment area mostly hold administrative positions, but find themselves within organisational cultures where innovation, excitement, novelty, technology and materialistic values are emphasised (often referred to as adrenalin cultures).
  5. In general, the younger candidates obtained the highest scores on the more complex processes including that of Logical Reasoning and Complexity. Could it be, that when challenged, the younger candidates generally perform better compared to the overall norm group?
  6. This was not the case for the older groups who seemed to perform best on the Verbal Conceptualisation, Memory and Gradual Improvement competencies. Again, this finding points towards a reliance of the older group on existing knowledge and experience, probably combined with the confidence to express oneself creatively and coherently. The older group achieved significantly higher scores on Gradual Improvement Learning than on Quick Insight Learning – which may indicate lower levels of cognitive flexibility.
  7. Given the often vague and unfamiliar nature of fast changing business environments, the information processing competencies of Exploration and Judgement are of particular interest. Within dynamic contexts, facts are not always readily available, and the decision maker has to rely on their intuition. The Judgement IPC measures that intuitive tendency by tracking: (a) awareness of vague and fuzzy information; (b) exploring this appropriately but not excessively; (c) clarifying vagueness using intuition; and (d) contextualising the decision.
  8. The younger candidates as a whole – especially those from the Technical/Engineering/Research group also obtained the highest Exploration IPC scores (which indicates effectiveness) while they did not necessarily show an Explorative style (indicating preference). In other words, they may not have explored as much as other groups, but their Exploration processes were more effective. In addition, it seems that younger candidates generally also achieve higher scores on Judgement than the older groups. Even though the older groups relied more on their intuition than younger candidates, their reliance on knowledge and experience was not as effective in unfamiliar contexts as the Judgement of the more flexible younger candidates.
  9. The younger candidates from the Creative/Media group, obtained the highest Explorative style scores (indicative of preference, not capability), yet their effectiveness of Exploration (IPC scores) were relatively low when compared to that of their peers from different career groups. Of all those from the 20 – 30 year old age group, those in the Creative/Media category also obtained the lowest scores on the Rules and Categorisation IPC skills, plus they also did not capitalise much on their knowledge and skills by effectively applying their Memory skills. It seems a popular strategy for organisations to adopt a culture which emphasise creativity and exploration (or adrenalin cultures), to attract and accommodate Millennials.
    Organisations which implement such a strategy, in the absence of carefully monitored initiatives to balance the excitement with an emphasis on structure and rigour, may do so at their peril. It may result in superficiality in the pursuit of new opportunities, without the necessary follow through to ensure sustainability. An excessively creative culture may thus stimulate explorative endeavours which may be ineffective and a waste of time and effort.
  10. The ineffective and superficial cognitive tendencies of the younger sample in the Creative/Media group, however, seems to improve over time in that individuals become slightly more Reflective with age. Their Complexity, Verbal Conceptualisation and Logical IPC scores also seem to improve over time and later even surpass that of other groups such as those from the Sales, HR and Administrative fields.
  11. The next question which comes to mind is how the various groups performed in terms of the Intuitive style. With age, people generally become more intuitive – given a reliance on their well-developed knowledge and experience bases. They may also become more integrative in that they increasingly make sense of their worlds. The results here indicate that the older group – the Marketing/Sales/Service group in particular, achieved the highest average score on the use of an Intuitive style. The older candidates also achieved the lowest average IPC scores on Rules, Categorisation and Judgement. This may show their increasing reliance on Intuition given their well-internalised experience- and knowledge-bases – a skill which may not necessarily generalise to dealing with unfamiliar information. (Note that the CPP measures Judgement capability in dealing with unfamiliar information.)
  12. Although the older group was relatively on a par with some of the younger career groups in terms of Memory use, the exercise of this skill may also not necessarily be effective in unfamiliar environments where previously acquired knowledge and skills do not apply. With the CPP assessment, the capacity to concentrate, retain and recall new and unfamiliar information, is measured. Although the older candidates may generally rely more on their memories, there may be factors which impact on their retention of new information, such as stress, overload, demotivation and possible cognitive rigidity and thus, a loss of concentration due to these aspects.
  13. The younger groups – the Technical/Engineering/Research group in particular, capitalised least on their Intuitive style. Instead, and as could be expected, they achieved the highest average scores on Logical, Analytical and Reflective stylistic tendencies. It is an important prerequisite for the technical and financial groups to be accurate – thereby reinforcing the application of a detailed, rigorous and reflective approach.
  14. It is interesting that the technical and financial groups, most of whom hold multiple degrees, tend to be marginally more logically than integratively inclined. The Integrative style is one which often characterises theorists who are faced with discrepant information or conflicting models and who enjoy interpreting these in a coherent and meaningful manner. The Logical style is more rule-based, factual and rigorous and aimed at identifying downstream effects and consequences. Systems thinking, a prerequisite for strategy formulation, however, largely requires an Integrative and Holistic cognitive approach.

It should be pointed out once again that a great diversity of cognitive approaches can be identified within each of these age and career groups. The measurement and contextualisation of individual differences thus remain crucial to optimise person-job matching, employee engagement and the strategic viability of the organisation.