Citation Distortions > Unfounded Authority

How Citation Distortions Create Unfounded Authority: Analysis Of A Citation Network / Steven A Greenberg / Associate Professor Of Neurology

Children’s Hospital / Informatics Program and Department of Neurology / Brigham and Women’s Hospita / Harvard Medical School / 75 Francis Street / Boston MA / 02115 / USA / sagreenberg@partners.org

BMJ 2009;339:b2680 / Published 21 July 2009, doi:10.1136/bmj.b2680

Objective

To understand belief in a specific scientific claim by studying the pattern of citations among papers stating it.

Design

A complete citation network was constructed from all PubMed indexed English literature papers addressing the belief that β amyloid, a protein accumulated in the brain in Alzheimer’s disease, is produced by and injures skeletal muscle of patients with inclusion body myositis. Social network theory and graph theory were used to analyse this network.

Main outcome measures

Citation bias, amplification, and invention, and their effects on determining authority.

Results

The network contained 242 papers and 675 citations addressing the belief, with 220 553 citation paths supporting it. Unfounded authority was established by citation bias against papers that refuted or weakened the belief; amplification, the marked expansion of the belief system by papers presenting no data addressing it; and forms of invention such as the conversion of hypothesis into fact through citation alone. Extension of this network into text within grants funded by the National Institutes of Health and obtained through the Freedom of Information Act showed the same phenomena present and sometimes used to justify requests for funding.

Conclusion

Citation is both an impartial scholarly method and a powerful form of social communication. Through distortions in its social use that include bias, amplification, and invention, citation can be used to generate information cascades resulting in unfounded authority of claims. Construction and analysis of a claim specific citation network may clarify the nature of a published belief system and expose distorted methods of social citation.

Source And Full Text Available At

HTML [http://www.bmj.com/cgi/content/full/339/jul20_3/b2680]

PDF [http://www.bmj.com/cgi/reprint/339/jul20_3/b2680]

News Coverage

Diversion, Invention, and Socialized Medicine

[http://scholarlykitchen.sspnet.org/2009/07/30/diversion-invention-and-socialized-medicine/]

From Publishing to Knowledge Networks: Reinventing Online Knowledge Infrastructures

Alexander Hars / Berlin: Springer/ 2003 / ISBN 3-540-01250-8 / $US 104

Today’s publishing infrastructure is rapidly changing. As electronic journals, digital libraries, collaboratories, logic servers, and other knowledge infrastructures emerge on the internet, the key aspects of this transformation need to be identified. Here, the author details the implications that this transformation is having on the creation, dissemination and organization of academic knowledge.

The author shows that many established publishing principles need to be given up in order to facilitate this transformation. The text provides valuable insights for knowledge managers, designers of internet-based knowledge infrastructures, and professionals in the publishing industry. Researchers will find the scenarios and implications for research processes stimulating and thought-provoking.

Source

[http://www.springer.com/business/business+information+systems/book/978-3-540-01250-4]

Content Overview

1. Leveraging information technology for science 1
1.1. Motivation 1
1.2. Analytical focus 5
1.3. Objectives 7
1.4. Approach 7
2. Characteristics of scientific knowledge infrastructures 9
2.1. Theoretical analysis 10
2.2. Empirical analysis: Emerging knowledge infrastructures 34
2.3. Visions of scientific knowledge infrastructures 55
2.4. Synthesis 57
3. Structure of scientific knowledge 83
3.1. Objectives 83
3.2. Theoretical foundations 87
3.3. Object-oriented model of scientific knowledge 102
3.4. Elements of scientific knowledge 124
4. Implications 187
4.1. Feasibility: IS Cybrarium 187
4.2. Conclusion 196

Source and Detailed Table Of Contents

[http://www.springer.com/cda/content/document/cda_downloaddocument/9783540012504-t1.txt]

[http://www.springer.com/cda/content/document/cda_downloaddocument/9783540012504-t1.pdf]

Google Books

[http://tinyurl.com/nye3aa]

Review

[http://tinyurl.com/l83e3c]

The Scientific Paper in the Age of Twitter

Walter Benjamin and Biz Stone

The FASEB Journal / 2009 / 23 / 7 / 2015-2018

Biz Stone: (dryly) In the middle of what?

For centuries a small number of writers were confronted by many thousands of readers. This changed toward the end of the last century. It began with the daily press opening to its readers space for 'letters to the editor.' And today... at any moment the reader is ready to turn into a writer.

Maureen Dowd: If you were out with a girl and she started twittering about it in the middle, would that be a deal-breaker or a turn-on?

Maureen Dowd: Why did you think the answer to e-mail was a new kind of e-mail?
Biz Stone: With Twitter, it’s as easy to unfollow as it is to follow.
—The New York Times, 2009 (1)

—Walter Benjamin, 1931 (2)

All registered users are able to add Notes, Comments, and Ratings to any article...Highlight the text to be annotated, and then click the 'Add a note to the text' link in the right-hand navigation menu of the article ...Notes can be started at any point within the text, but for ease of reading we ask that you do not begin Notes in the middle of words.
—Public Library of Science, 2009 (3)

THE SANDBOX OF IDEAS

It’s reassuring to read that our colleagues at The Public Library of Science have remained true to the integrity of the word, if not the sentence or thought. PLoS One has raised this banner for verbal integrity in a cheery commercial entitled "PLoS Journals Sandbox: A Place to Learn and Play (3) ." The new format, which permits instant interruption of on-line, formal scientific papers, is certainly in keeping with the temper of our time. Were this to have been the practice in old-fashioned print libraries, many of our journals would by now resemble kitty litter.

In the Age of Twitter we’ve become accustomed to bell-tones and roving thumbs in every venue of human life. We call it social networking when we summon up Facebook, YouTube, or MySpace—and it’s no longer limited to teenagers. Twitter and the other social networks have been used by nearly one in five of online adults ages 25 to 34 (4) . Nowadays, in the plenary sessions of national scientific meetings, one sees heads bowed in homage to the Holy Book of Face or tweeting to Twitter in fewer than 140 characters of text.

Biz Stone, the founder of Twitter, explains:

Twitter is a service for friends, family, and co–workers to communicate and stay connected through the exchange of quick, frequent answers to one simple question: What are you doing? (5)

[snip]

And as for science: what are we doing? Today, on screens large and small, every online scientific paper is just a cursor stroke away. That makes it possible, as Benjamin predicted, for any reader to turn into a writer. No surprise, then, that PLoS and other new venture journals encourage us to adorn the digital text with notes and comments, blogs and tweets. [snip] Right on to the Public Library of Science! How fitting it is that PLoS, the youngest kid on the block of reputable science journals, is out to compete in the sandbox of ideas (3) .

ENDANGERED SPECIES OF PRINT

It’s no secret that scientific journals have been losing readers of their printed versions to the greater audience on the web. For many scientific journals, the number of "hits" they receive daily online is a factor or two greater than their monthly print circulation. [snip]The printed word still retains a good chunk of older devotees, but even these are as likely as their younger colleagues to prefer electronic to printed copies of their favorite journals (8) . [snip]

This sea change in the way that information is handled and supported has worried many and frightened a few (9) . We might recall that scientific journals as we know them are relatively recent arrivals on the scene and have moved along paths trod by the general culture. [snip]. Science and publishing became professionalized at the dawn of the Enlightenment. The two oldest scientific journals on record are The Philosophical Transactions of the Royal Society (London) and the Journal de Scavants (Paris), both founded in 1665. Originally filled with material of general interest for fellow citizens of "the republic of letters," they soon morphed into publications that reported the most rigorous science of the day (10) . [snip]

IT HAS NOT ESCAPED OUR NOTICE

The mold was struck for the modern scientific paper between the two world wars. [snip] .Today the acronymic IMRaD formula (Introduction, Methods, Results, and Discussion) is now required by all reputable journals, including this one. But there’s always been wiggle-room around the canonical IMRaD format; most journals are enlivened by letters to the editor, rebuttals, conference proceedings, abstracts of meetings, news reports, etc. Walter Benjamin’s description in 1931 of the marketplace of print still applies to the market in scientific ideas:

Today there is hardly a gainfully employed European who could not, in principle, find an opportunity to publish somewhere or other comments on his work, grievances, documentary reports, or that sort of thing. Thus, the distinction between author and public is about to lose its basic character (13).

He would have loved texting and Twitter; I can imagine his pleasure at running his thumbs over the passing comments and pertinent grievances as he "follows" and "unfollows" as both author and reader

In this context, one can only imagine what the epochal Watson-Crick paper would look like these days on PLoS. Their 1953 paper was written as a "Letter to the Editor" in Nature and never underwent peer review. John Maddox, editor-in-chief at the time, later admitted that "the Crick and Watson paper could not have been refereed: its correctness is self-evident." That’s a matter of dispute, as we’ll see (14) . The Watson-Crick paper begins with:

We wish to suggest a structure for the salt of deoxyribose nucleic acid (D.N.A.). This structure has novel features which are of considerable biological interest...

The ending of the paper is of course perhaps the best known in scientific prose:

It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material.

But for many of us, the real action is in the acknowledgments at the end:

We are much indebted to Dr. Jerry Donohue for constant advice and criticism, especially on interatomic distances. We have also been stimulated by a knowledge of the general nature of the unpublished experimental results and ideas of Dr. M. H. F. Wilkins, Dr. R. E. Franklin and their co-workers at King’s College, London (15).

One need only to imagine what tweets, twoops, formal corrections, and comments might decorate these passages on PLoSOne today. Pauling, Chargaff, Avery, Meselson, Cairns, Donohue, Perutz, Franklin, and Wilkins would have had their say:

This structure has novel features COMMENT: YEAH! HYDROGEN BONDING, LINUS!) which are of considerable biological interest. COMMENT: FOR WHICH I WROTE THE CHEMISTRY, ERWIN FORMAL CORRECTION: IT’S THE GENETIC MATERIAL, YOU FOOLS!, GENES! OSWALD

It has not escaped our notice that the specific pairing FORMAL CORRECTION: BASE PAIRING A/T=G/C, ERWIN we have postulated immediately suggests a possible copying mechanism for the genetic material. COMMENT: LIKE WHAT? CONSERVED? SEMI? MATT COMMENT: MORTAL OR IMMORTAL? CAIRNS

We are much indebted to Dr. Jerry Donohue for constant advice and criticism, especially on interatomic distances. FORMAL CORRECTION: SEZ YOU! I TOLD YOU ABOUT THE KETO TO ENOL TAUTOMERS. YOU KNEW SQUAT FROM THE CHEMISTRY! JERRY We have also been stimulated by a knowledge of the general nature FORMAL CORRECTION: I SHOWED YOU THEIR PICTURES, MAX of the unpublished experimental results and ideas of Dr. M. H. F. Wilkins, Dr. R. E. Franklin FORMAL CORRECTION: YOU PEEKED, "DARK LADY" and their co-workers at King’s College, London COMMENT: OUR TWO FOLLOWING PAPERS ARE DATA, YOURS IS A LEAP, MAURY.

ARCADES TO THE BORDER

Walter Benjamin, (1895–1940) the quintessential European intellect and literary omnivore, would have loved having a COMMENTS and FORMAL CORRECTIONS option at his finger-tips. [snip]

More to the point: much of the Arcades Project prefigures the home page of a social network on the Web. Benjamin literally explores a network: the linked indoor shopping arcades of nineteenth century Paris, the Passages (16) . I imagine a Benjamin today, reincarnated as the perennial flaneur; who follows a path in the Arcade of Panoramas. He stops occasionally at one site or another site. The flaneur ambles (surfs) along a protected space (MySpace) in which bustling crowds are reflected in shiny Windows. He adjusts his cravat in a store-front mirror (Facebook), and when the bell-tone rings in his pocket, he takes out his timepiece (Blackberry). He looks past his mirror image (YouTube), to find two generations of followers (Twitter).

Were Benjamin to log on to Twitter, he’d have thousands of tweets on hand to send to generations of followers. [snip]In the century of the common man film was art without "aura" and accessible to all:

Magician and surgeon compare to painter and cameraman. The painter maintains in his work a natural distance from reality, the cameraman penetrates deeply into its web...Thus, for contemporary man the representation of reality by the film is incomparably more significant than that of the painter, since it offers, precisely because of the thoroughgoing permeation of reality with mechanical equipment, an aspect of reality which is free of all equipment (2) .

I can see Benjamin now tweeting, now twoopsing, now blogging, now surfing, now scrolling. His thumbs move quickly over the tiny keys—the sandbox of images in sight. He tweets directly to Biz Jones and the other followers of WB (his nom-de-tweet), an upbeat quote from Paul Valery (1928). Valery and WB were sure that other great gadgets would soon supplant celluloid film:


Pretty good prediction, no? Isn’t that "simple movement of the hand" what the thumbs are doing these days on a Blackberry. The quote is also about twice the 140 characters that Biz Stone permits, but heck, WB could have split it in two.

[snip]

It’s less than 140 characters. I’d bet that Benjamin would have been at home in our new world of texting and tweets, blogs and hand-helds. In the Age of Twitter, he’d be ready to play in the sandbox of ideas, and we wait for his FASEB Journal essay in "Milestones."

Source and Full Text (Open Access?)

Full Text Available

HTML [http://www.fasebj.org/cgi/content/full/23/7/2015]

PDF [http://www.fasebj.org/cgi/reprint/23/7/2015]

Just as water, gas, and electricity are brought into our houses from far off to satisfy our needs in response to a minimal effort, so we shall be supplied with visual or auditory images, which will appear and disappear at a simple movement of the hand, hardly more than a sign (17) .

viXra.org: An Alternative Archive Of e-Prints In Science And Mathematics

ViXra.org is an e-print archive set up as an alternative to the popular arXiv.org service owned by Cornell University.

[http://vixra.org/]

It has been founded by scientists who find they are unable to submit their articles to arXiv.org because of Cornell University's policy of endorsements and moderation designed to filter out e-prints that they consider inappropriate.ViXra is an open repository for new scientific articles. It does not endorse e-prints accepted on its website, neither does it review them against criteria such as correctness or author's credentials.

Source

[http://vixra.org/]

Why viXra?

In 1991 the electronic e-print archive, now known as arXiv.org, was founded at Los Alamos National Laboritories. In the early days of the World Wide Web it was open to submissions from all scientific researchers, but gradually a policy of moderation was employed to block articles that the administrators considered unsuitable. In 2004 this was replaced by a system of endorsements to reduce the workload and place responsibility of moderation on the endorsers. The stated intention was to permit anybody from the scientific community to continue contributing. However many of us who had successfully submitted e-prints before then found that we were no longer able to. Even those with doctorates in physics and long histories of publication in scientific journals can no longer contribute to the arXiv unless they can find an endorser in a suitable research institution.

The policies of Cornell University who now control the arXiv are so strict that even when someone succeeds in finding an endorser their e-print may still be rejected or moved to the "physics" category of the arXiv where it is likely to get less attention. Those who endorse articles that Cornell find unsuitable are under threat of losing their right to endorse or even their own ability to submit e-prints. Given the harm this might cause to their careers it is no surprise that endorsers are very conservative when considering articles from people they do not know. [snip]

[snip]

It is inevitable that viXra will therefore contain e-prints that many scientists will consider clearly wrong and unscientific. However, it will also be a repository for new ideas that the scientific establishment is not currently willing to consider. Other perfectly conventional e-prints will be found here simply because the authors were not able to find a suitable endorser for the arXiv or because they prefer a more open system. It is our belief that anybody who considers themselves to have done scientific work should have the right to place it in an archive in order to communicate the idea to a wide public. They should also be allowed to stake their claim of priority in case the idea is recognised as important in the future.

[snip]

In part viXra.org is a parody of arXiv.org to highlight Cornell University's unacceptable censorship policy. It is also an experiment to see what kind of scientific work is being excluded by the arXiv. But most of all it is a serious and permanent e-print archive for scientific work. Unlike arXiv.org it is truly open to scientists from all walks of life.

Source

[http://vixra.org/why]

News Coverage

Fledgling site challenges arXiv server

[http://physicsworld.com/cws/article/news/39845]

Rejecta Mathematica: Caveat Emptor

Rejecta Mathematica is a real open access online journal publishing only papers that have been rejected from peer-reviewed journals in the mathematical sciences.

[http://math.rejecta.org/]

About Rejecta Mathematica

Mission

Rejecta Mathematica is an open access, online journal that publishes only papers that have been rejected from peer-reviewed journals (or conferences with comparable review standards) in the mathematical sciences.

At Rejecta Mathematica we believe that many previously rejected papers (even those rejected for legitimate reasons) can nonetheless have a very real value to the academic community. This value may take many forms:

• "mapping the blind alleys of science": papers containing negative results can warn others against futile directions;


• "reinventing the wheel": papers accidentally rederiving a known result may contain new insight or ideas;


• "squaring the circle": papers discovered to contain a serious technical flaw may nevertheless contain information or ideas of interest;


• "applications of cold fusion": papers based on a controversial premise may contain ideas applicable in more traditional settings;


• "misunderstood genius": other papers may simply have no natural home among existing journals.

All research papers appearing in Rejecta Mathematica include an open letter from the authors discussing the paper's original review process, disclosing any known flaws in the paper and stating the case for the paper's value to the community.

Selection and Scope

Rejecta Mathematica publishes two types of papers: research articles and correspondences. The screening process for publishing research articles in Rejecta Mathematica includes no technical peer review (hence the slogan Caveat Emptor); rather, papers are selected on the basis of their potential interest to researchers in the mathematical sciences.

It is expected that the authors will discuss any known flaws or rediscoveries with full and honest disclosure in their open letter. As an additional means of quality control, follow-up correspondences are strongly encouraged from the community at-large and will be considered for subsequent publication.

The scope of Rejecta Mathematica is very broad, encompassing all disciplines relating to the mathematical sciences, including: pure and applied mathematics, statistics, engineering, and computer science.

Open Access

Rejecta Mathematica is an open access journal; all papers appearing in Rejecta Mathematica are immediately made freely available via this website for downloading, reading and distributing as long as the original authors and source are attributed.

All works published in Rejecta Mathematica are distributed under a Creative Commons Attribution Non-commercial license that allows redistribution of the work while the authors retain copyright ownership. Read more about Rejecta Mathematica copyright policies, including the specific terms of the distribution license.

Other Information

Rejecta Mathematica is published by Rejecta Publications, Inc., a non-profit corporation based in Texas. Rejecta Publications, Inc. is not affiliated with any university or other educational institution.

Donations are graciously accepted in order to help defray the (modest) cost of web-hosting and administrative expenses. Alternately, you can support Rejecta Publications by purchasing Rejecta Mathematica merchandise.

Other Rejecta journals may follow (in disciplines outside the mathematical sciences). Please contact us if you are interested in starting your own Rejecta franchise.

Source

[http:/math.rejecta.org/about-rejecta-mathematica]

Frequently Asked Questions

[http://math.rejecta.org/frequently-asked-questions]

Reinventing Academic Publishing | 1 > 2 > 3 / James Hendler

Letter from the Editor / Intelligent Readers

PART ONE

Although quoting yourself is generally considered tacky, I've been involved in several recent activities and discussions I'd like to share with you. These largely arose from "Publishing on the Semantic Web," a column that Tim Berners-Lee and I coauthored in Nature back in 2001 (www.nature.com/nature/debates/e-access/Articles/bernerslee.htm). In that column, one of a series of opinion pieces about academic publishing's future, we discussed the Semantic Web's potential impact. We ended with this somewhat brash statement:

The semantic web will provide unifying underlying technologies to allow these concepts to be progressively linked into a universal web of knowledge, and will therefore help to break down the walls erected by lack of communication, and allow researchers to find and understand products from other scientific disciplines. The very notion of a journal of medicine separate from a journal of bioinformatics, separate from the writings of \ physicists, chemists, psychologists and even kindergarten teachers, will someday become as out of date as the print journal is becoming to our graduate students.

In the past year, I've found that this quote is resonating more and more and that some of the big players in academic publishing are starting to think along these lines [snip]. Not all of them are considering using Semantic Web technology; some are inspired instead by Web 2.0's community-oriented features. However, the notion of breaking down the lines between traditional disciplines and reaching out to audiences beyond the academic bench scientist is becoming an important "meme" in academic publishing.



More Than Just Technology



One leader in this area has been Nature itself, which launched the Nature Network [http://network.nature.com/], a social-networking and blogging site aimed primarily at scientists. Users can create a social network, share forums and blogs, and use tags to create semantics. [snip]



Lately, I've been hearing from other publishers and magazines that they're also considering doing more to enhance their online sites with community-oriented features. They have been motivated by Nature's lead, by users' increasing reluctance to pay for hardcopy articles when so much is free online, and by the increasing facility that young scientists, [snip]



It's becoming clear that making a successful community-oriented Web site requires more than just the technology. [snip]While ... [some] sites have taken off, others have languished owing to misunderstanding the mismatch between the technology and the community they want to reach.



Overcoming Resistance



Which brings us back to academic publishing. As publishers try to promote new models of communication among scientists, with an eye toward finding some new role in the process, they need to respect the way science works. Although this, like many other things, might be changing owing to the Web's impact, some natural points of resistance must be overcome before new, more community-oriented, interdisciplinary scientific sites succeed. While scientists have gloried in the Web's disruptive effect on publishers and libraries, with many fields strongly pushing open publication models, we're much more resistant to letting it disrupt the practice of our disciplines.



At the Science Foo Camp (tagged on many blogging sites as "scifoo") held at Google in August 2007, several sessions dealt with academic publishing's future. The topics included open-source publishing; publishing "pre-review" (or with community reviewing of some sort); and the use of blogs, wikis, and other new technologies to enhance scientific communication.



However, motivated by comments arising the first day, the second morning featured the session "Culture of Fear: Scientific Communication and Young Scientists." This session, led by postdoctoral researchers Alex Palazzo (Harvard) and Andrew Walkingshaw (Cambridge), explored issues that those starting out in scientific fields face when using these new technologies. The job market for scientific positions, especially in academia, is tight. So how do a team of scientists, sharing partial results pre-publication, assign credit? Authorship blurs when small amounts of information, which might contain key insights into making processes successful, are publically shared. How does a blogger get credit for the information that leads to an eventual publication at a competing lab?



Another theme of the session was peer review's role in scientific fields. Although some pseudoscientists have claimed that we use peer review to keep their brilliant insights out of our precious literature, most scientists truly appreciate the filter that peer review provides. The high standards publications maintain are a useful way to ensure that ideas are well argued and strongly evaluated before being published. On the other hand, some feeling has always existed in the community that, especially with respect to funding, the peer review process might be overly constraining and considerably delay new ideas from coming to the fore. [snip] These factors have motivated many in the community to discuss new mechanisms, based on emerging Web technologies, that let us communicate more ideas more quickly.



For example, one model involves publishing after a minimal peer review and then creating some sort of postpublication metrics as to the paper's value. Some online journals are already exploring this model. However, as Alex put it in his [http://scienceblogs.com/transcript/2007/08/scifoo_day_3_well_that_was_yes.php],

Until the scientific establishment reaches a consensus as to whether these post-publication metrics are indeed useful for determining the credentials of a scientist in the shorter term ... .

What Can We Do?



So, we arrive at the crux of the issue facing many of us, whether we're the editor in chief of a magazine such as IEEE Intelligent Systems or the head of a major press or publishing house. How do we embrace the new technology and encourage more of the sharing that Tim and I were calling for, without causing career risk to the very people to whom the technology is most familiar—the younger scientists? If we don't think through the social issues of usage, the technologies alone won't have any significant impact and will go largely unused.



One option—and I'd like to see more effort in this area—is for innovation to come "from the top." Eventually, as these young scientists become the leaders of our fields, they will bring these new technologies with them. But with the world in its current shape, needing the help of scientists for our very existence, we can't afford to wait that long. Rather, we need to find ways to bring more senior scientists into contact with the positive side of these technologies.



In computer science, where the barrier has been lower than in some fields for senior people to learn to use new computer technologies, we've seen some of this happen. For example, Tim Finin has been instrumental in bringing bloggers to the AAAI conference, which has caused others to read, and in some cases create, blog content. [snip] When young scientists see their field's leaders embracing new technologies, it's that much easier for them to demonstrate to the rest of us, without fear of retribution, what these technologies can do.



Conclusion



It's time for us as computer scientists to take a leading role in creating innovation in this area. Some ideas are simple—for example, providing overlay journals that link existing Web publications, thus increasing the visibility (and therefore impact) of research that cuts across fields. [snip]



In my next column, I'll discuss current ideas regarding new technologies for academic communication that we as a field might be able to help bring into being, and some of the obstacles thereto. I look forward to hearing your thoughts on the subject.



Full Text Available At



IEEE Intelligent Systems, Vol. 22, No. 5. (2007), pp. 2-3.

[http://www2.computer.org/portal/web/csdl/doi/10.1109/MIS.2007.93]

Letter From The Editor / Interlligent Readers

PART TWO

Last issue, my letter focused on some trends in academic publishing that journals, magazines, and other scientific-publishing endeavors are facing. I argued that we computer scientists should take a leading role in helping create technologies that will break down the walls between different disciplines.

An obvious response is, "haven't we already?" That is, Tim Berners-Lee's invention of the World Wide Web, coupled with Sergey Brin and Lawrence Page's PageRank algorithm that powers Google, has led to a world where scientists publish preprints and papers on the Web and can find each others' results without having to subscribe to the same journals. Doesn't that solve the problem? Just pick your favorite chemical, check it out on Google (or even better, Google Scholar), and there's the paper you need. What more is there to say?

New Medium, Old Ways



Unfortunately, the reality of sharing papers on the Web doesn't live up to this ideal. With rare exceptions, Web-based journals and open source publications and preprint servers have been modeled on the same field and subfield considerations that print journals and scientific communication have been using for decades. [snip]



So while we've made journals electronic, with positive results on distribution, we still haven't really done much to revolutionize the scientific-communication aspects of scientific publication. If a computer scientist searches for some term in the field—say, "case-based reasoning"—in an online journal, he or she will likely find papers of interest. [snip] Despite the prevalence of online literature and the ability to search with great ease, jargons—the different ways different fields describe similar things—still get in the way.



This is true even of sites that have tried hard to provide organizational structures to help people find each other's work. Scientific publication sites such as PubMed [http://www.ncbi.nlm.nih.gov/sites/entrez] or arXiv [http://arxiv.org/] , for example, organize papers on the basis of disciplinary terms familiar to the scientists who use them. [snip]



This is particularly troubling because many, if not most, of the key scientific discoveries of the next generations will require interdisciplinary approaches. [snip] Unfortunately, despite the growth of these larger science projects, the publishing still tends to be in traditional journals, arranged along disciplinary lines and reviewed by experts in the traditional fields. [snip]



Overlay Journals



This need for better support for interdisciplinary work is exactly what's motivating many of the new solutions being considered online. Rather than simply replacing print journals with online journals of the same ilk, people are considering new models that can more easily cross disciplines and find papers (or projects and so on) related to their interest areas. One model that's becoming more common is the overlay journal (sometimes called a deconstructed journal, after John Smith's 1997 paper "The Deconstructed Journal"). The idea is to create an online publication by taking some crosscutting theme and providing links to papers published elsewhere. By providing links rather than republishing, the overlay journal provides a service to both the reader, by linking to many publications, and the publishers, by bringing more eyeballs to their sites. [snip]



Models of Curating



A number of different models are being explored for how best to "curate" such overlay publications. One model is to have the overlay journal function as an independent publication. The IEEE Computer Society will be taking this approach with Computing Online, a site that will serve multiple purposes. One purpose is to provide an overlay functionality across the different Computer Society magazines ... . The new publication will thus provide a means for a wider audience to see our magazine's articles, and we'll be providing articles to that site, reducing its need to solicit and produce new material. [snip]



Another model has a more open Web feel to it. For example, the University of Southampton's Leslie Carr is creating a submitter-based Web science overlay journal. Submitters can fill in a simple form (or share metadata from other publishers) to have a paper listed on the site. The submitter

• points to the original publication,
• proposes where to link it into an evolving hierarchy of terms (basically a folksonomy seeded by a set of terms from a domain ontology, thus providing Semantic Web metadata), and
• briefly comments on why to include this paper.

The submitter can be the paper's author or someone else who feels the paper will be of value to the community. Mechanisms for determining how to moderate the site ... are still being discussed. This mix of submission with overlay seems to combine the best of several worlds.

Conclusion

[snip]. Beyond the overlay journal, we start to look at more novel ideas that depart further from traditional publishing. Such ideas include blog-based scientific publication, Web 2.0-based community sites, and sites enhanced by the Semantic "Web of data." More on this in part 3.

Full Text Available At



IEEE Intelligent Systems, vol. 22, no. 6, pp. 2-3, Nov./Dec. 2007



[http://www2.computer.org/portal/web/csdl/abs/html/mags/ex/2007/06/mex2007060002.htm]

Letter From The Editor / Intelligent Readers

PART THREE

[snip]



In this last part of my editorial trilogy, I discuss something that's becoming more and more important to academic communication—online scientific interaction outside the traditional journal space.



The importance of contextWe must change our focus from scientific disciplines to scientific "contexts." When looking at the most successful Web technologies, especially in what's known as Web 2.0, we see that many of the most exciting sites exploit a community or context focus. [snip]



[snip]



The Power of Social Networking



Social websites for particular communities of scientists offer a way to, essentially, embedding the context into the site. A good example of this is myExperiment.org, a social-networking site for experimental scientists. The site lets users share experimental workflows and develop communities around specific activities. A network of scientists can form, for example, around certain proteins' role in causing disease. These scientists can share methodologies and specialized techniques and discuss how to get better results in a relatively informal and blog-like way, while still being able to share the important work products that help them be more effective. While these scientists might still compete with respect to the data they're using and their published results, they can cooperate in developing and refining experimental methodologies. This crucial sharing of knowledge typically isn't publishable in traditional journals.



A more generalized version of this idea, and one I'm coming to rely on in my own work, is the Twine system, developed by radarnetworks.com. [snip]



Twine is a social-networking site that focuses not on its members' activities per se but on sharing information products in user-created contexts. [snip]



In part 2 of this editorial trilogy I talked about overlay journals and how they could be used for sharing information across an emerging discipline. A problem is that if the group wanting to communicate is relatively small, setting up such a site involves a prohibitive amount of time and resources. Using something like Twine, we can create sort of an ad hoc overlay journal on our specific topic of interest. Twine wasn't created with scientific communication in mind, but something like this is obviously applicable to scientific discourse. The twine forms the context, and the members choose what to share as they create their own shared vocabulary. Unlike email, newsgroups, and even wikis, the open and dynamic nature of building the discourse in the social-networking context could greatly improve such information sharing.



The Source of Change



I'm excited by the emerging technologies for scientific interaction; these new technologies that expand on blog- and wiki-like ideas to create context mechanisms will become increasingly important to scientific discourse. However, whether change in online scientific communication comes specifically through technologies such as these or through new Web technologies just starting to be explored, I'm certain of one thing: this change is inevitable.



The other day, I was reading a blog entry by my friend and coauthor Dean Allemang [http://dallemang.typepad.com/my_weblog/2007/12/what-will-2008.html[. He says that when he suggests to people in management that their enterprise could use something such as a blog, wiki, or other self-organizing information space, they reply, "You just don't understand. Our engineers / researchers/analysts will never do that. It just won't happen!" I know this phenomenon well; I've heard it myself when I make similar suggestions to my academic colleagues about the use of technologies such as myExperiment or Twine.



Dean goes on to say, however, that there's a "new generation of people entering the workforce who have a different relationship with Wikipedia than their elders. [snip]. They share information as a natural part of their lives; why in the world wouldn't they do the same in their work contexts?



He makes a good point, one that's also true in academia. These same Wikipedia college students are becoming our graduate students and tomorrow's scientists. Just as a previous generation of students created the scientific websites that have become crucial to our daily work, this new generation is using emerging technologies to create mechanisms for sharing their interests. I hope those of us who hold more senior positions will find ways to encourage and endorse this work and reward their efforts. But even if we don't, the change will come. Frankly, I think that's a great thing.

Full Text Available At

IEEE Intelligent Systems, vol. 23, no. 1, pp. 2-3, Jan./Feb. 2008,

[http://www2.computer.org/portal/web/csdl/doi/10.1109/MIS.2008.12]

Reinventing Academic Publishing Online. Part I

Volume 14, Number 8 / 3 August 2009

Abstract

While current computing practice abounds with innovations like online auctions, blogs, wikis, twitter, social networks and online social games, few if any genuinely new theories have taken root in the corresponding “top” academic journals. Those creating computing progress increasingly see these journals as unreadable, out-dated and irrelevant. Yet as technology practice creates, technology theory is if anything becoming even more conforming and less relevant.

We attribute this to the erroneous assumption that research rigor is excellence, a myth contradicted by the scientific method itself. Excess rigor supports the demands of appointment, grant and promotion committees, but is drying up the wells of academic inspiration.

Part I of this paper chronicles the inevitable limits of what can only be called a feudal academic knowledge exchange system, with trends like exclusivity, slowness, narrowness, conservatism, self-involvement and inaccessibility.

We predict an upcoming social upheaval in academic publishing as it shifts from a feudal to democratic form, from knowledge managed by the few to knowledge managed by the many.

The technology trigger is socio-technical advances. The drive will be that only democratic knowledge exchange can scale up to support the breadth, speed and flexibility modern cross-disciplinary research needs. Part II suggests the sort of socio-technical design needed to bring this transformation about.

Contents

• The role of academic knowledge exchange
• Feudal knowledge exchange trends
• Cross–disciplinary research
• Conclusions

The Role of Academic Knowledge Exchange

Introduction

Caveat lector: Previous iterations of what you’re about to read have been dismissed by information systems (IS) editors and reviewers since a first draft written in 1999 after an ISWorld rigor/relevance discussion. Many years of rejection confirm it as unpublishable in IS. This seems partly because high–level papers always have faults, and partly because suggesting to his tailors that the emperor of academic publishing is wearing only the fig leaf of rigor is unwise. If you find the academic publishing system “excellently attired” please read no further, as here we argue it has serious problems that need addressing. Yet our target is not the many good authors, wise reviewers and supportive editors in our field, many of whom are personal friends.

Our target is the feudal knowledge exchage system they currently work under. While academia covers many disciplines, our evidential case is the field of technology use — the reader must judge for themselves in their field. Yet as much the same case has been made in the field of quantum physics (Smolin, 2006) our conclusions may benefit others.

Part I argues that the current gate–keeping model of academic publishing is performing poorly as knowledge expands and interacts, and that academic publishing must reinvent itself to be inclusive and democratic rather than exclusive and plutocratic. Part II suggests a design to do this using already successful socio–technical tools.

Knowledge exchange systems

[snip]

While science may once have consisted of amateurs cultivating private knowledge gardens, today it is organized into specialist fiefdoms that defend themselves vigorously. Academics are now gate–keepers of feudal knowledge castles, not humble knowledge gardeners. They have for over a century successfully organized, specialized and built walls against error. However the problem with castles, whether physical or intellectual, is that they dominate the landscape, they make the majority subservient and apathetic, and battles for their power reduce productivity. As research grows, knowledge feudalism, like its physical counterpart, is a social advance that has had its day.

The theory–practice divide

[snip]

Each case had a grain of truth, but for technology use the predictive power of theory has been low and the gap between theory and practice is widening. In Eric Raymond’s (1997) analogy, the bazaar of technology practice is booming while the cathedral of technology theory is declining, because one is open and one is closed.

Bridging the divide

[snip]

Yet creating a new online global society is a socio–technical system as complex as any space program, as socio–technical systems need both social and technical performance to succeed (Whitworth and Moor, 2009c). We cannot expect to progress by trial and error alone. If theory and practice are the two legs of scientific progress, a crippled theory leg is a serious problem. We now suggest the main cause of this is unbalanced rigor.

The rigor problem

[snip]

We believe in rigor, but see system performance as a mix of many criteria (Whitworth, et al., 2008), which “bite back” if one criteria is exclusively pursued at the expense of others (Tenner, 1996). The better model of knowledge exchange performance is of an efficient frontier — a line of many points that defines the best one can get of rigor given a value of relevance (Keeney and Raiffa, 1976). Pursuing rigor alone produces rigor mortis in the theory leg of scientific progress.

The role of research

If excess rigor reduces innovation and causes theory to lag behind practice, in IS at least, why not change the strategy? Surely academics prefer to ride the technology wave rather than struggle along behind it?

[snip]

When a system becomes the mechanism for power, profit and control, idealized goals like the search for truth can easily take a back seat. Authors may not personally want their work locked away in expensive journals that only endowed western universities can afford, but business exclusivity requires it. [snip]

[snip]

One can justify distributing rare economic resources to the few, as there is not enough to go around, but one cannot justify distributing knowledge this way, as giving knowledge away does not diminish it. While physical resources distribute by a zero–sum model, information resources follow a non–zero–sum model (Wright, 2000), where the more one gives the more synergy is created (Whitworth, 2009a). Economic scarcity is no argument for knowledge exclusivity.

Conformity training

The modern academic system has become almost a training ground for conformity. PhD students spend three–six years as apprentices under senior direction, then another three–six years seeking the security of a tenured appointment. At both stages, criticizing the establishment is unwise if one wants a career. It is not surprising that six–12 years of such training produces people who toe the party line.[snip]

[snip]

Due to publishing pressure senior IS leaders explicitly advise new faculty not to innovate if they want a career! As the word “unfortunately” suggests, they take no responsibility for a system that actively drives innovators out to make their breakthroughs in practice, e.g., the movement of automatic indexing from universities to commercial enterprises like Google (Arms, 2008).

Changing the system

Can this system change itself? IS academics traditionally judge journal importance by measures like internal expert perceptions, number of citations and publication numbers (Hamilton and Ives, 1982). These internally generated and self–reinforcing measures all favor the status quo. As an academic publishing review notes: “What gives this enterprise its peculiar cast is the fact that the producers of knowledge are also its primary consumers.” [6]

Current research into journal quality illustrates the contrast between science as a search for gain and science as a search for truth. While accepting that “science can be perceived as a social network which accumulates, distributes and processes new knowledge” [7], they see journal “quality” in terms of stakeholder gains:

1. So authors can publish in quality journals (for better career impact);


2. So readers can select quality journals (to save time);


3. So tenure and promotion committees can choose staff (more easily); and,


4. So libraries can more easily choose quality publications [8].

The analysis contains no mention of the community good of uncovering the truth, or of any reality beyond individual gains. “Quality” is assumed to equate to rigor ... .

Yet, as argued, equating quality with rigor is an error, as quality needs both rigor and relevance. When academia incestuously rates itself by citation studies and expert ratings it can easily become a self–reinforcing system disconnected from external reality (Katerattanakul, et al., 2003).

The IS case

[snip]

[snip] There was a major strategic failure of vision and leadership in IS, as a growing academic discipline should be a melting pot of new ideas, not a stagnant pool of old ones.

How rigor constricts

Even respected IS journal editors recognize there is a problem: “Research publications in IS do not appear to be publishing the right sort or content of research.” [13] The cause we suggest is social conformity to old theories. [snip]

[snip]

The problem lies not with “old but good” theories but with a system that seems unable to grow new ones around them. Given the enormous changes of the last decade in computing, the lack of matching theoretical innovation over the same period is nothing short of astounding.

[snip]

The reality is that it is hard to publish a new theory in mainstream IS, if “new” means not an old theory tweak and “theory” means more than speculative conjecture. Innovation is not a term that comes to mind as one reviews technology use theory yet in technology use practice precisely the opposite is true. That progress is coming from practice — not theory — suggests that theory has its priorities wrong.

Feudal Knowledge Exchange Trends

We have described a feudal knowledge exchange system run by the few for the few, supported ideologically by the church of rigor, financed by university factories of knowledge, whose goal is to dominate and defend the purity of specialized intellectual fiefdoms. We now outline some inevitable trends of such a system, again for the IS case.

Exclusive

[snip]

The trend is for a few exclusive top journals to dominate the theoretical landscape. The alternative proposed in Part II, is a more democratic system.

Outdated

A KES is outdated when its information flows mainly address issues that are no longer current. Lack of timeliness due to publication delay is a Type II opportunity loss. What use is quality that is too late to affect things, when others have either solved or bypassed the problem?

[snip]

The rigor justification that truly good papers will end up published somewhere, so nothing is lost by Type II errors is simply not true. In the glacial world of academic publishing one rejection can delay publication by two–four years. Of the good papers rejected, some despair, some move to greener pastures, but most just conform to reviewer “suggestions”. If rejectees do not try again, publishing delayed, like justice, is publishing denied, as some leave academia for good [snip].

Conservative

A KES is conservative if it resists change and innovation. A rising rigor bar means that new theories face a greater burden of proof than old ones (Avison, et al., 2006). That new theories respect the old is reasonable, but when they face critiques that old theories don’t answer either, then those who have climbed the tree of knowledge have pulled the ladder up behind them. New theories rarely rise like Venus from the sea, fully formed and faultless. Usually new ideas begin imperfect and only develop over time with help from others. So if anything, the bias should be the other way. When new theories must be fully proved before they can even be proposed as research questions, then we have got science backwards.

As Einstein is said to have said: “If we knew what we were doing, it wouldn’t be called research, would it?”

[snip]

Authors who innovate risk their careers, as even their successful innovations may not flourish until after their tenure decision. It should not be this way. Innovators are the “whistle blowers” of academia — they challenge false claims of knowledge profits. A system that rejects its own agents of change rejects its own progress.

[snip]

New ideas by definition contradict the agreed norm, so can be expected to polarize reviewers. A proposal that offends no one probably changes nothing. Yet in academic hiring one bad reference can kill an appointment [18], and in journal submissions and grant proposals, a “perfect” application must get a perfect score not one person must dislike it. Yet if no one dislikes your work you probably aren’t doing anything worthwhile. Indeed a hallmark of innovation is that it polarizes people — some love it and some hate it. The score tick box system of most grant reviews weeds out creativity.

A hundred years ago Einstein invented special relativity working in the Swiss Patent Office because no university would appoint him. Yet he revolutionized physics. Is the academic system today any more inviting to unorthodoxy? [snip]

Part II explores how to change this.

Unread

[snip]

If the democratic KES outlined in Part II lets everyone publish, won’t that worsen the not–reading problem, as there will be more to read? It would — if the motivation didn’t change, but it will. While in a risk–avoiding system more papers are more error to avoid, in a value–seeking system more papers are more potential value. Readers will use electronic tools, like Google Scholar, to do positive searches. While the literature seems huge, a search on a specific research topic may produce only a handful of relevant papers. Even imperfect papers may have good parts or stimulate new ideas. When the motive moves from following normative ideas to finding useful knowledge, more people will read a greater variety of papers.

The opposite of apathy is involvement and participation, and in Part II we suggest that socio–technical tools can turn readers from passive recipients of pre–selected “quality” to active participants in value generation.

Inaccessible

A KES is inaccessible when most of its potential users cannot write to it or read it.

In academia, to contribute one must pass the reviewer firewall. [snip] The rigor trend predicts negatively driven reviewing based on denying faults rather than growing value. In contrast the democratic KES outlined in Part II can report review contributions and still respect anonymity, which increases incentives for quality reviewing.

Specialized

[snip]

As more rigorous and exclusive “specialties” emerge, the expected trend is an academic publishing system that produces more and more about less and less. The alternative proposed in Part II is to tear down the walls to instead allow more and more about more and more.

The end point

Under a rigor trend top journals will be exclusive in participation, innovation averse, few in number, outdated in content, restricted in scope, largely unread and increasingly specialized. Authors will duplicate, imitate and supplicate rather than innovate. They will recycle old theories under catchy new labels, develop minor “tweaks” to gatekeeper theories and never rock the boat of received opinion. Reviewers will deny, critique and oppose author attempts to publish while readers will graze, skim and browse the old ideas in new clothes that get through — if they read them at all.

The feudal answer to more people writing is more rejections and more people not reading. The expected end point will be journals that are more rigorous than relevant, authors more prolific than productive, reviewers denying not inspiring, and readers grazing but not digesting. The reader can decide if this applies to their field.

This final vision of journals as exclusive and isolated castles of specialist knowledge, manned by editor–sovereigns and reviewer–barons, raising the barricade of rigor against a mass assault by peasant–authors seeking tenure knighthoods, is not inspiring.

The worry that opening the gates of the knowledge citadel will let in a flood of error confuses democracy with anarchy. Government by the people does not mean no rules, it just means new rules. It does not destroy hierarchies, just opens them to all by merit. To the academic realists now playing the publishing game, this is “the way it is”, and ideas of knowledge democracy are unreal idealism. Yet the same would have been said of physical democracy in the middle ages. Social change emerges as individuals evolve.

The cracks in the current system are already showing ... . A democratic knowledge economy will outperform its feudal equivalent for the same reason that democratic physical economies outperform feudal ones — that people produce more when control is shared.[snip]

Cross–Disciplinary Research

In multi–disciplinary research academic specialists work side by side on the belief that specialty ideas will cross–fertilize, but increased specialization reduces this likelihood. In contrast cross–disciplinary research uses faculty trained in more than one discipline to merge knowledge across specialties. [snip]

The nexus of technology use

We identify cross–disciplinary research at the nexus of technology use as an area of knowledge expansion. Terms like Web science (Fischetti, 2006), socio–technical systems (Whitworth and Moor, 2009c), information communication technology (ICT), information systems, social computing, information science, informatics and Science 2.0 (Shneiderman, 2008) all point to a nascent “knowledge flower” growing at the crossroads of technology use (Figure 1).

[snip]

Figure 1

[snip]

Cross–disciplinary crunch

As the number of knowledge specialties increases the number of cross–disciplinary connections grows geometrically. The two–dimensional Figure 1 cannot illustrate this, as with eight disciplines there should be 256 potential overlaps, not just eight as shown. A science with hundreds of distinct disciplines has tens of thousands of knowledge intersections, each potentially another specialty.

[snip]

Knowledge expansion at the intersection of disciplines is a chance for evolutionary progress, rather than a sign of failure. Building walls to protect knowledge is necessary in a land of bandits and thieves, but in a land of earnest artisans it only reduces beneficial synergies, and forces each specialty to reinvent the intellectual wheels of others.

[snip]

Conclusions

1. The demands of cross–disciplinary research suggest that academia should:
2. Replace the myth that rigor is excellence with research as a risk–opportunity mix;
3. Reduce business influence on the grounds that academic truth is good business; and,
4. Reinvent academic publishing as a democratic open knowledge exchange system.

Socio–technologies like wikis show what is possible when communities activate, but wikis are not the academic answer as they don’t attribute or allocate accountability, nor offer anonymous review. The easy options in academic publishing have already been tried, so Part II of this paper suggests a socio–technical hybrid.

A democratic KES would reaffirm academia’s original goal of publishing knowledge freely for mutual critique and benefit. The search for knowledge should be open not closed, dynamic not static, inclusive not exclusive, current not outdated, affirming not denying, innovative not conservative and most of all, living not dead. To achieve this goal academics must hold to the goal of knowledge growth. If we do our duty as others do theirs, progress will occur naturally. Lest academia forget, its very reason to exist is to grow knowledge, not to guard it, nor to profit from it.

Notes

References

This paper is licensed under a Creative Commons Attribution–Noncommercial–Share Alike 3.0 United States License

Source and Full Text

[http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2609]

NOTE: Pre-Print Of Part II > SOCIO-TECHNICAL VISION Self-Archived PDF Full Text Available At

[http://brianwhitworth.com/BWRF-FM-Part1.pdf]