The following article is adapted from a section produced for a deliverable within the PALETTE project.
In the complex relationship between technology development and technology use, the continual emergence of new technologies and new ways of using technology in the wider context necessarily has an impact on development and use of technology in complex R&D projects. In an earlier work (Nodes and axes of change in the way technology is used in learning) we explored axes of change in technology seen in terms of the way technology is used in learning. Two questions arose as a result of that work. The first concerned the mechanisms that exist within R&D projects to keep track of these changes. The second was about how this knowledge is integrated into R&D projects, particularly in the form of technical modifications to tools and services being developed. Informal discussions with technical leaders within the PALETTE project following on from the circulation of the article mentioned above pointed to an additional question: what are the differences in perception of research and development on the part of those working on technology and what influence do project structures and ways of working have on the balance between the two within a project. The following article explores these three questions on the basis of interviews with seven key figures of technological development in the PALETTE project.
For those working in computer sciences, technology watch is perceived as an integral part of their work. As an activity that is largely seen as self-evident, it is not necessarily formalised although it may be part of a well-tried routine. Neither the process nor the results are generally written down.
Technology watch is invariably an individual activity. Amongst possible sources of information, the following were mentioned: journals, conferences, Web sites, blogs, word of mouth, chance meetings, being a member of committees or networks, belonging to mailing lists, using RSS feeds, reviewing articles for scientific journals, … Much technology watch is enriched by informal exchange. The interviews seem to indicate that there is a culture of exchange about new ideas amongst those working on technology. When some people talked of technology watch as a function of a team rather than the individual, they were referring more to the sharing of results than the work of being on the look out for new ideas. The trans-disciplinary nature of a project like PALETTE was seen to particularly favour the exchange of the results of technology watch. In some cases, technology watch can be undertaken systematically, for example when working on a particular deliverable. Technology watch for a specific deliverable is clearly anchored in a given project, but most technology watch is not necessarily carried out on a project-by-project basis but overflows as a background activity going beyond the boundaries of projects. One person called it the “glue” between projects.
Integrating unforeseen new ideas
Let us now turn to the processes by which a project capitalises on results emerging from technology watch. Here we are neither talking of changes due to requests from users about technology being developed nor about changes due to external dependencies, like, for example, the impact of modifications of browser technology on web-based services.
As we have seen above, technology watch is not generally specific to a given project. This implies that additional steps need to be taken to pinpoint those recent research results and emerging technologies that are promising for a particular project but which did not exist when the project was written or where not considered pertinent at the time of writing. Here too there is often no formal procedure although there are informal practices, like, for example, presenting a new idea at the end of a weekly project meeting. When it comes to “minor features”, there are no formal processes for reaching decisions about adjusting existing features or adding new ones. As one person put it: “In the area of Web development, people rarely vote. It is more a question of consensus building and the subtle influences of individuals.” Sometimes choices are made by the individual developer within the scope of his or her remit. Other times, decisions result from demonstrations and discussions amongst team members and may lead to formal decisions taken by the work package leader. The PALETTE project had a specific work package dedicated to the “integration” between technological development and the evolving practices of users. Several interviewees stressed the importance of its role as the ‘heart of the project’, a place where consensus could be reached on technical developments. Occasionally major questions were addressed to the overall Steering Committee for a decision.
Two factors have a particular influence on the adoption or adaptation of unforeseen technical ideas in the project thanks to technology watch:
- The impetus of individual initiatives;
- The overarching structure of contractual agreements like the initial Description of Work document (DOW) and the subsequent versions of the Implementation Plan (IP).
The uptake depends on the actions of individuals who have an idea and who implement it so as to demonstrate it to others. A key strategy in getting across a new idea is to make it visible in meetings through demonstrations and discussions. And a key success factor in seeking the uptake of a new idea, as one person put it, is “the power to convince of the person driving the idea.”
In addition, the openness of project participants towards the integration of unplanned ideas from technology watch depends on the flexibility of contractual documents that govern the project and how participants perceive them. When talking about the integration of new ideas during the interviews, many people spoke of the IP and the DOW. They were said to be written in such a way as to leave room for change and development. The initial formulation of the project was referred to by several people as being a challenge. It is interesting to note that projects are not necessarily put together in optimum conditions. Yet that initial formulation and the related negotiation between partners are going to condition the outcomes of the whole project, despite the fact that periodic revisions of the implementation plan allow for adjustments.
The relationship between individual initiatives and collective objectives was put in a slightly comical way but one interviewee: “you have to find the right balance between ‘agitation’ and perseverance in the initial direction.” An additional tension exists between the ambitions and agendas of the different groups involved in the project and the objectives of the consortium as expressed in the official project documents.
The dynamic between research and development
Why look at the dynamic between research and development? Because the differing conceptions of the two and the relationship between them as perceived by those people involved in the technology development strongly condition the working of a project especially when there is an ambition to include users more actively in the development process.
When asked to explain the difference between research and development, a number of explanations were put forward. One such explanation opposed both the generic and the specific and the short term and the long term, stating that the aim of research was to improve understanding and to draw up generic solutions whereas development sought to create solutions to satisfy immediate needs often in a specific context. Another opposed the unknown and the unmapped with a precise set of specifications, research being an exploration in which the solutions are not necessarily known whereas development involves coding according to given specifications. Finally another explanation contrasted knowing and doing in which research sought to understand the world to be able to “master” it by producing knowledge in the form of models, whereas development had to do with producing artefacts using the knowledge developed by research. One thing that was stressed in this last perspective was that both the acts of knowing and doing could be seen as scientific if they incorporated appropriate methods to validate and verify models and artefacts. The perspective of knowing and doing throws a particular light on the relationship between research and development in which the latter builds on the former. One person estimated that there could be up to ten years delay between the two when it came to key questions, implying that any major research in short-term projects was unlikely to be used in the same project.
All the people interviewed expressed a clear preference for research over development, with one person saying that the optimum ratio of research to development should be about 80% – 20%. At the same time almost all interviewees felt that there was too much development in the project with the ratio being closer to 50% – 50%. Various reasons were put forward for this perceived imbalance. One was the overall framework of which the project was a part as a European Commission funded integrated project that necessarily required more development to produce distinct products. But the main reason evoked for the predominance of development over research was the role of users in the project: the PALETTE project evolved a participative methodology designed to favour on-going exchange between users, pedagogues and developers. One person explained that this participatory design methodology (PDM) constantly placed the user in the development process in such a way that it created short-term pressure on technical teams to produce results. As a consequence many of the interviewees spoke with distaste of being forced into the role of suppliers to users – a couple of people even talked evocatively of “slaves” – having to develop solutions rather than carry out research. According to them, efforts in PALETTE were concentrated on satisfying Communities of Practice (CoPs). Some people felt that CoPs didn’t sufficiently understand the purposes and challenges of the project. At the same time, the technical actors didn’t necessarily understand the need to remodel the development process as epitomised by the PDM. In the software engineering cycle, which people considered as well established, normally users were only consulted right at the beginning and at the end of the process. Technical actors couldn’t see the point of further involvement of users. With time, according to one interviewee, the developers have come to assimilate some of the ideas of the PDM into their development process even if they may not use the same words for things as the pedagogues.
When asked to what extent project structures influenced work on research and development, interviewees had differing answers. These included praise for the interdisciplinary approach as it was felt to increase exchange between a wider variety of disciplines, opening new perspectives. Working with an implementation plan (IP) was also seen as positive as it kept research work on track, but the formulation of those IPs was seen as excessively time consuming to the detriment of research. Preparing deliverables was also seen to be to the detriment of time available for research, although deliverables were also praised as occasionally raising interesting debates. One of the major criticisms, however, of the project structures as far as the relationship between research and development was concerned pointed to problems of timing and approach in the initial phase of the project. The pedagogues were to produce use cases on the basis of which the participative design process was supposed to be built. This took time. During this period many of the developers, not understanding the point of the PDM in relation to their work and their approach to development, were rearing to go, knowing exactly what they intended to develop and how to develop it. In addition, several interviewees mentioned that the technical work did not start from scratch at the beginning of the project, but was built not only on existing experience and prevailing research agendas, but also sometimes on existing tools. As a result, the technical work began out of phase with the pedagogical work and had to be subsequently adjusted.
The implications for R&D projects
Crafting the starting point
The conditions for drawing up complex U funded projects are not optimum, yet the starting point constituted by the initial project description, despite subsequent mechanisms to periodically adjust that formulation, heavily determines the success of the project. A number of strategies might improve that situation. First and foremost, funders might consider preselecting a small number of potential projects on the basis of shorter project outlines and then fund a working meeting of project partners to draw up project proposals. Another strategy, indicated by what goes above, would be to encourage a more holistic approach to understanding the state of art during the formulation of the project going beyond the narrower disciplinary perspectives on what is current and what is not. At the same time, more attention needs to be granted to external dependencies and developing strategies to react to potential risks and changes. Finally, project partners need to carefully consider the appropriateness of taking forward technologies and approaches from earlier projects, weighing up the potential gains in time and effort against the loss of flexibility and responsiveness to the needs of the project and their fellow partners.
Room for change and unpredictability
Suitable mechanisms need to be found to improve the integration of input from wider technology watch into complex projects, in particular encouraging the circulation of new ideas and making more explicit the decision-making processes about the pertinence and possible uptake of these new ideas.
An integrative, trans-disciplinary approach can play a key role in developing exchange and the potential spread of new ideas from technology watch by providing structured contact between actors from different backgrounds. Additional work needs to be done to further develop ways of favouring individual initiatives and collective discussions about outputs from technology watch.
In addition, for such innovation to be possible, room needs to be made for on-going change at all levels of project organisation. One of the implications of this approach would be the need for funders of projects such as the European Commission to allow greater flexibility in the nature of deliverables and to accept a certain degree of unpredictability in project outcomes. Another implication would be the need for a renewed approach to project management that is able to handle uncertainty and keep track of and negotiate change and innovation without necessarily discouraging it.
Between research and development
The tension between research and development perceived in the discourse of technological actors raises questions about the suitability of engaging researchers when the process, from a user perspective, requires concrete, functioning results. In addition, the heavy emphasis put on research over development by the technological actors raises serious questions not only about the feasibility of developing viable ‘products’ for users but also about the sustainability of any software developed in the context of such a project. CoPs are extremely reticent about investing time and effort in the up-take of IT tools that will not be maintained and continue to be developed after the end of the project. In contrast, researchers do not perceive the maintenance and the further development of such ‘products’ as being part of their research remit. Perhaps the answer lies in finding a suitable balance between computer science research activities and technical development that fits the needs of the project for concrete reliable products.
Alan McCluskey, Saint-Blaise.