How communication success can be measured
The study first clarifies which variables research uses to measure the success of professional communication, and then asks the contractors of two internationally active agency networks about these variables. The result is narrower than the initial assumption. Of three success parameters and eight indicators, two remain in the final model, the trust of the contractor’s customers and a gain in reputation.
Three perspectives instead of two
Marketing research usually judges success from two directions, from the customer’s side and from the value of the company. For agency work this falls short, because many messages are not aimed at a sale at all. Image work, public relations and the communication of social commitment act above all on a company’s standing. The study therefore adds a third perspective, that of society. It is based on an evaluation of 24 empirical studies on communication success, which are listed in Appendix 3 of the thesis.
Each of the three success parameters stands for one party in the communication network of agency, contractor, the contractor’s customers and the public.
- Customer value. The benefit experienced by the contractor’s customers. It shows in their attitude, their loyalty and their trust.
- Social value. The benefit that the public perceives, for instance when a company credibly communicates social or ecological commitment.
- Financial value. The economic result at the contractor, that is, at the sender of the message.
According to the literature, the three variables are related. Customer value and social value are supposed to be reflected over time in firm value, similar to what marketing research describes for the chain from customer value via brand value to firm value. This chain is laid out in the literature model of the success parameters. The study did not estimate it: in the survey, communication quality acts on each of the three variables individually, and the model contains no paths between the success parameters.
How research has measured success so far
Customer value
Customer value research treats customer relationships as an asset in which a company must invest. Measurement is mostly by surveys, because satisfaction, attitude and buying intention cannot be observed directly. Satisfaction arises from the comparison between expected and received performance. If it lasts, loyalty grows from it, and from both trust forms over a longer period. Trust rests on the assumed credibility, reliability and honesty of the partner.
For an influence of communication on customer value, the study evaluates several empirical studies. In a survey of 383 consumers, brand awareness, attitude, preference and buying intention were positively related. In 30 DAX companies, perceived competence correlated positively with sales, whereas perceived likeability correlated negatively. From this, the study adopts the three indicators attitude, loyalty and trust.
Social value
Social value concerns effects that go beyond the product, usually in connection with sustainability. The studies evaluated show that the effect depends on credibility. Commitment that customers regard as honest strengthens loyalty to the company. Communication perceived as dishonest deters. Only a small group of critical consumers buys consistently sustainably, but this group acts as opinion leaders. In a survey of employees in the USA, around 79% stated that they preferred to work for a socially committed company. From these studies the study derives four indicators, namely public interest, reputation and image gains, employee motivation and a reduced business risk.
Financial value
For the financial contribution of communication, the literature knows four calculation methods. The net asset approach estimates what rebuilding the communicative structures would cost. Stock value approaches look at the gap between stock value and net asset value. Tobin’s q relates market capitalization to net asset value; a value above 1 points to intangible potential that the market prices in. Discounted cash flow methods calculate the present value of the revenues attributed to a communication measure.
All four methods have the same problem. Financial results occur with a delay, and competition, the business cycle or stock market sentiment overlay the contribution of an individual measure. The empirical findings are correspondingly mixed. In 200 companies from 40 industries, customer satisfaction and Tobin’s q were related. An evaluation of 41 car manufacturers, by contrast, found that marketing raises sales in the short term and lowers firm value in the long term. The study adopts three key figures that are established in marketing studies, the stock value or market value, the return on equity (ROE) and Tobin’s q.
Eight indicators after the pre-test
Two indicators of social value were below the total mean of 1.70 in the pre-test, at 1.17 and 1.58, and were discarded. Employee motivation would have captured employee topics twice, because the employees are already on the enabler side in the model. The reduced business risk was unclear to several respondents, and some saw expensive sustainability investments rather as an additional risk. The financial key figures remained in the model, although the respondents described them as speculative and measurable only in the long term.
| Success parameter | Indicator | Pre-test relevance (0 to 3) | Survey mean (1 to 5) | High or maximum consent |
|---|---|---|---|---|
| Customer value | Attitude | 2.08 | 3.69 | 64% |
| Customer value | Loyalty | 2.08 | 3.80 | 64% |
| Customer value | Trust | 1.75 | 3.75 | 63% |
| Social value | Public interest | 1.83 | 3.43 | 53% |
| Social value | Reputation and image gains | 1.75 | 3.55 | 58% |
| Financial value | Stock value or market value | 1.83 | 3.16 | 40% |
| Financial value | Return on equity | 1.83 | 3.19 | 43% |
| Financial value | Tobin’s q | 2.00 | 3.14 | 39% |
The relevance values come from the pre-test with agency executives; the means and shares come from the survey of 209 contractors.
What the contractors report
Each success statement referred to a specific, completed project and was rated from 1 (no consent) to 5 (maximum consent). The evaluation covers 209 complete questionnaires.
The contractors see customer value as having improved most. The means lie between 3.69 and 3.80, and 64% agree highly or fully that the attitude of their customers has improved. For social value, the means are 3.43 and 3.55. Financial value performs weakest. Its means are around 3.15, and between 28% and 31% of the respondents see no or only slight effects. The study itself points out that satisfaction surveys tend to produce favourable judgments. The absolute values are therefore to be read with caution.
What remains in the model
Means do not yet show a relationship with the quality of the agency. This is tested by the structural equation model (SEM). In it, the four enablers of the QMC model, the quality management model for the communication industry, act on the three success parameters via a mediating variable, communication quality. Three of the seven core hypotheses concern the success side. The path to customer value (H5) is set as a reference path and is counted as accepted by the study. The path to social value (H6) is significant at the 10% level (p = 0.08). The path to financial value (H7) is not significant (p = 0.24); the hypothesis is rejected. At the level of the individual indicators, no success parameter reaches significance on its own in the initial model.
In three amendments, financial value, public interest, customer loyalty and customer attitude dropped out; customer trust and reputation gains remained as direct success indicators (Results).
- Customer value, measured by customer attitude, customer loyalty and customer trust.
- Social value, measured by public interest and reputation gains.
- Financial value, measured by stock or market value, return on equity and Tobin’s q.
- Only customer trust and reputation gains remained in the final model; all other indicators were dropped.
In the final model, customer trust has a standardized weight of 0.80, and reputation gains have one of 0.14 (p = 0.098). The path to this result is notable. In the confirmatory factor analysis (CFA), trust was barely related to the construct customer value, for which it was intended as an indicator. In the final model, trust is the success indicator most closely related to communication quality. Attitude and loyalty, rated highest in the pre-test, dropped out of the model.
Why the financial key figures are missing
That financial value disappears contrasts with a large part of the marketing literature, which reports positive relationships between communication expenditure and financial key figures. The study names several reasons. Financial success depends on factors that communication can hardly influence: competition, the overall economic situation and innovative capacity. The survey also refers to agency work, which often does not aim at the sale of a specific product and whose financial consequences can hardly be observed in the short term.
A characteristic of the design adds to this. The financial effect, too, was collected as an assessment by the respondents and not calculated from balance sheet or share price data. The study did not examine whether trust and reputation nevertheless raise the contractors’ competitiveness later on. This question is on the list of open research questions.
Success in communication is not marketing success
From the findings the study derives three differences from the marketing perspective.
| In marketing, success counts as | In communication work, success shows as |
|---|---|
| specific buying intention | building and maintaining customer trust |
| short-term attention to products or promotions | long-term reputation of the company |
| immediate increase in firm value | intangible effect and continuity |
For the assessment of agency work, this means that, according to this study, its effect is to be found rather in the trust of the contractor’s customers and in the contractor’s standing than in quarterly figures. Anyone who gauges the success of a communication project only by returns or stock value is measuring at the point where the contractors surveyed saw the least effect.
QMC model
Unlike the EFQM, the model measures financial value directly and lists employees only as an enabler.
Method
After the pre-test, 20 of the 26 measurement categories from the literature remained for the questionnaire.
Results
After three amendments, a final model with three enablers and two success parameters remains.
Leadership
In the final model, leadership efficiency is the strongest enabler, with a standardized weight of 0.98.
Practice
The thesis concludes that transparency in agencies arises more from informal exchange than from QM routines.