Internal and External Validity of the Vacancy Database
| Autor | Jeisson Cárdenas Rubio |
| Páginas | 192-235 |
192 C
A WEB -BASE D APPRO ACH TO M E A SU R E S KILL MISM ATCHE S AND S KILL S PROFI LES FO R A DEV ELOPI NG COUN TRY
8. Internal and External Validity
of the Vacancy Database
193
c
8.1. Introduction
The previous c hapter described the main cha racteristics of the Colombian
vacancy database from 2016 to 2018. However, these results do not provide
enough evidence about t he validity or reliab ility of vacancy data for add ressing
unemployment and i nformality problem s in Colombia. As is the case w ith data
collected wit h other methods (e.g. survey s), the data collected from onl ine sources
have some caveats that might affect the i nterpretation of results. Compan ies
can post wrong or cont radictory in formation; for instance, employers might
request an engineering professional with just a high school diploma or a full-
time enginee ring professional with a n extremely low salar y. Moreover, errors
in posted infor mation might arise f rom data mining process es. The algorithms
created in the previous chapters m ight fail. For insta nce, the algorithm that
looks for patterns in job description s might confuse some words, and incor-
rectly create va riables of a universit y degree or job experience, a mong others.
Consequently, errors or biases might ar ise in the infor mation and af fect the
internal a nd external consiste ncy of the vacancy database. T hus, this chapter
tests the inter nal and external validity of the vaca ncy infor mation.
Internal va lidity refers to t he consistency of the var iables within t he vacancy
database (Henson 2001; Streiner 2003). In ideal condition s, the results f rom
a variable in t he vacancy database should not contrad ict the findings from other
variables in t he same database; otherw ise, the results will be u nreliable. One
straightfor ward way to addr ess this issue is to compare the results of differ-
ent but related var iables. Therefore, the second section of th is chapter tests
the interna l validity of t he vacancy databa se via cross tabulations and wage
distribution analysis.
Internal va lidity is a cr ucial aspect to consider before drawing any con-
clusions on labour demand from the vacancy database. Establishing result
consistency fr om the vacancy database wit hin a particular e conomic context
C c
194 C
A WEB -BASE D APPRO ACH TO M E A SU R E S KILL MISM ATCHE S AND S KILL S PROFI LES FO R A DEV ELOPI NG COUN TRY
(external va lidity) is another relevant factor to consider before drawing any
conclusions about Colombia’s labour ma rket (Kureková, Beblavy, and Thu m,
2014). Exter nal validit y, specifically, refers to pos sible biases or representa-
tiveness issues in the data (Ra smussen 2008; Stopher 2012).
Logically, all sou rces of informat ion have limitation s. For instance (as
mentioned in Chapte r 4), in Colombia, the current sect oral surveys car ried out
by the DANE do not provide det ailed information about human capital, such
as occupational structu re or the skills required in each position. Web-based
informat ion might help to fill this gap. However, the on line sources utili sed for
the database in this study a lso have limitat ions.
Given the nature of these online sou rces, job vacancy information m ight
describe a particular segment of the labour market. The ex ternal validity of
results depends on which kind s of vacancies are bei ng published online for
the countr y of interest. To test external val idity, it is necessary to process a nd
compare the res ults from other sourc es of information (e.g. household su rveys)
with the vaca ncy database result s. Therefore, Section 3 d iscusses the represe n-
tativeness of the Colombian vacancy database by categorising t he household
labour sur vey (GEIH) accord ing to ISCO-08 categories, as wel l as comparing
the Colombian vacanc y data set with official nat ional labour stat istics.
Additional ly, Sections 2 and 3 propose a framewor k to evaluate the repre-
sentativeness of the vacancy database for each occupation at di fferent levels
of disaggregation (e.g. four-digit ISCO level):
• When testi ng the internal consiste ncy of the information for a spec ific
occupation, are t here no errors or only minor errors?
• If yes, is the distribution of wages in the vacancy database for that
particu lar occupation si milar to the d istribution of wages in the
household sur vey?
• Can simila r seasonal trends be observed in t he level of employment
in the household survey and in the level of job vacancies?
• Can opposite seasonal trends be seen i n both the level of unemploy-
ment in the household su rvey and the level of job vacancies?
• Do lagged effects exist between the number of job adver tisements
and new hires?
Para continuar leyendo
Solicita tu prueba