How Far Did Vannevar Bush Get on the Memex?

Posted on July 26, 2026 by Brewster Kahle
Research report · Internet Archive holdings · 26 July 2026

Short answer: the memex itself was never built — not even a lab prototype. But Bush spent a decade building and funding its component machines, two of which were real, and one of which was delivered to the U.S. Navy. This report traces exactly how far each piece got, from sources in the Internet Archive’s own scanned collections.

§1What the memex was, and where it was only ever ink

Bush described the memex publicly just once during its era: “As We May Think,” in the Atlantic Monthly, July 1945 (vol. 176, no. 1, pp. 101–108) — a microfilm desk with two screens, a camera in the user’s hat-band, dry photography, and, at its heart, associative “trails” linking any two items. Two months later Life (Sept. 10, 1945, pp. 112–124) ran the condensed, illustrated version, with Alfred Crimi’s now-famous cutaway drawing of the memex desk. Life’s editors added the telling caveat that the machine was not yet in existence — a contemporary confirmation that in 1945 there was no hardware behind the essay.

The essay was not a sudden vision: a near-complete draft (“Mechanization and the Record”) circulated in 1939, and the concept traces to Bush’s MIT memos of the early 1930s. The drafts, and the two later memex essays, are collected in Nyce & Kahn’s From Memex to Hypertext (1991).

§2The machines he actually built

The Comparator (1937–38). Bush’s first information machine was built not for libraries but for Navy cryptanalysis (OP-20-G): a microfilm-and-photocell device to compare message tapes at high speed. It was delivered — and it failed. Colin Burke’s archival history records that when the components finally reached Washington they did not fit together, and reliability problems (film shrinkage, counters, optics) dogged it into storage. Burke’s verdict is that the Comparator’s failure shadowed Bush’s later information-machine efforts.

The Rapid Selector (1938–40). This is the closest the memex ever came to existence: a working MIT prototype that spun reels of 35 mm microfilm past a photoelectric reader, matching binary dot-codes beside each frame and flash-copying the hits — memex’s storage, coding, and retrieval mechanics, minus the trails. It was industrially sponsored (National Cash Register and Eastman Kodak money, on the order of $10,000-scale allocations Burke documents), engineered largely by Bush’s graduate students, and demonstrated. It never became a product, and war work ended it.

The descendants (1949–1950s). After the war Ralph Shaw, librarian of the U.S. Department of Agriculture, demonstrated a rebuilt Rapid Selector (engineered by Engineering Research Associates); intelligence agencies pursued their own — G. Pascal Zachary notes a 1950s NSA machine that was “really an elaborate version of the crude ‘rapid selector'” of the late 1930s. In practice the selectors were slow, fussy, and out-engineered by emerging digital methods; Burke quotes the 1950s judgment that “the fabled Bush Rapid Selector proved unworkable.” His book’s own machine inventory lists memex flatly as “machine never built.”

§3The late paper revisions

Bush redesigned the memex twice more — on paper. “Memex II” was drafted at his home in Belmont, Massachusetts after his 1955 retirement (manuscripts of 1958–59), swapping microfilm for magnetic storage and adding trails that strengthen with use; it went unpublished until Nyce & Kahn printed it in 1991. “Memex Revisited” (in Science Is Not Enough, 1967, at pp. 75 ff.) conceded that the enabling technology had finally arrived — and that the machine still had not been built. His memoir Pieces of the Action (1970) closes the record: Bush died in 1974 with the memex unrealized.

§4The verdict, stage by stage

  • Idea · reached 1932–39 Mechanized-library memos at MIT; the 1939 draft essay.
  • Published design · reached 1945 “As We May Think” (Atlantic; Life illustrated reprint). The memex never advanced past this stage as a whole machine.
  • Funded prototype · components only Rapid Selector at MIT (1938–40): microfilm store + coded search + photo-copying output, working in the lab. No trails.
  • Delivered system · components only Comparator, delivered to Navy OP-20-G (1938) — arrived broken, worked marginally, shelved. Postwar selectors (Shaw/ERA 1949, NSA versions) ran but disappointed.
  • Memex assembled · never No memex prototype was ever attempted, by Bush or anyone under him. The defining feature — associative trails — was never mechanized in his lifetime; it waited for digital hypertext (Engelbart and Nelson both credited the 1945 essay).
Why it stalled, per Burke and Buckland: Bush stayed committed to analog microfilm and photo-optics as computing went digital; dry photography and film handling never met the essay’s performance claims; and the memex’s social premise (a personal machine) had no sponsor in a decade of military patrons. Buckland adds the prior-art point: Emanuel Goldberg at Zeiss Ikon had patented and built a microfilm “statistical machine” by 1931 — so even the selector’s core mechanism was not, strictly, first.

§5Was it copyright that stopped him?

No — but the question has a real answer one movement over. In Burke’s archival history, copyright never appears as an obstacle to Bush’s machines: the Comparator failed on engineering, the Selector on reliability, funding, war priorities, and Bush’s analog commitments — and the only “infringement” in the record is patent infringement searching on the Selector (in the Honeywell v. Sperry-Rand files). His machines processed scientific abstracts and Navy cipher traffic; no publisher was ever in the loop to object. The memex itself never got far enough for rights to matter.

The publisher fight belonged to Robert C. Binkley and the documentation movement — the people whose project was access to the copyrighted record itself. Monika Dommann’s media history of copyright traces Binkley’s arc: early-1930s confidence that microfilm would let him rewrite the old copyright regime around “free trade in ideas”; the narrow truce of the 1935 Gentlemen’s Agreement with the National Association of Book Publishers (single copies for scholars, in place of a loan — while reaffirming the publishers’ exclusive rights); publishers questioning the agreement’s legal validity after the NABP reorganized in 1938; and Binkley’s growing disillusionment with the negotiations as the decade closed — turning against Berne ratification and insisting the right to copy is not the right to publish. He died in 1940 at forty-two, before the rematch that eventually arrived as Williams & Wilkins and CONTU.

Two ironies from the Nyce & Kahn volume: the only publisher friction in the whole memex story was over the essay — reprint-permission wrangling with LIFE, after several magazines had rejected the piece — and copyright enters the memex lineage as a design parameter only with Ted Nelson, whose “As We Will Think” (reprinted there) budgets royalties to copyright holders into the imagined system. The tradition eventually engineered for copyright; it was never stopped by it.

§6Where to verify each claim in the Archive’s scans

SourceItemWhere
Atlantic, July 1945sim_atlantic_1945-07_176_1 essay begins printed p. 101 (leaf n112), runs to p. 108
Life, Sept. 10 1945sim_life_1945-09-10_19_11 pp. 112–124 (leaves ~n113–n125); “not yet in existence” note near n115; memex-desk cutaway near n124
Burke 1994informationsecre0000burk thesis at n41; “proved unworkable” n23; “machine never built” inventory n29; Comparator delivery n270; selector schematics listed n12
Nyce & Kahn 1991frommemextohyper0000unse_b3u7 Memex II text and its Belmont drafting history at n134–n137; Burke’s “Career of the Rapid Selector” chapter (contents n8)
Bush 1967scienceisnotenou0000bush_p9y8 “Memex Revisited,” printed pp. 75 ff. (leaves n80–n86 carry the running heads)
Zachary 1997endlessfrontierv00zach selector origins n85–n87; NSA descendant n289
Bush 1970piecesofaction00bush memoir; career retrospective
Shaw 1949the-rapid-selector-shaw-1949 the postwar Selector by its operator; credits Bush’s “basic electronic system” and Goldberg’s 1931 patent on its first page
Dommann 2019authorsapparatus0000domm Binkley & the Gentlemen’s Agreement, “Celluloid Circulations” chapter, printed pp. 93–102 (n104–n121)
Gitelman 2014paperknowledge00gite Binkley chapter; his death in 1940 at n70
Binkley 1935sim_yale-review_1935-03_24_3 “New Tools for Men of Letters,” from printed p. 519 (n90)

§7Sources

  • Bush, Vannevar. “As We May Think.” The Atlantic Monthly 176, no. 1 (July 1945): 101–108. IA: sim_atlantic_1945-07_176_1. Also online at theatlantic.comarchive ↗ (Wayback-preserved).
  • Bush, Vannevar. “As We May Think” (condensed; illus. Alfred D. Crimi). Life 19, no. 11 (Sept. 10, 1945): 112–124. IA: sim_life_1945-09-10_19_11.
  • Burke, Colin. Information and Secrecy: Vannevar Bush, Ultra, and the Other Memex. Metuchen, N.J.: Scarecrow Press, 1994. IA: informationsecre0000burk. The standard archival history of the Comparator and Rapid Selector.
  • Nyce, James M., and Paul Kahn, eds. From Memex to Hypertext: Vannevar Bush and the Mind’s Machine. Boston: Academic Press, 1991. IA: frommemextohyper0000unse_b3u7. Reprints the 1939 draft, “Memex II” (first publication), and “Memex Revisited.”
  • Bush, Vannevar. “Memex Revisited.” In Science Is Not Enough. New York: Morrow, 1967. IA: scienceisnotenou0000bush_p9y8.
  • Bush, Vannevar. Pieces of the Action. New York: Morrow, 1970. IA: piecesofaction00bush.
  • Zachary, G. Pascal. Endless Frontier: Vannevar Bush, Engineer of the American Century. New York: Free Press, 1997. IA: endlessfrontierv00zach.
  • Buckland, Michael. “Emanuel Goldberg, Electronic Document Retrieval, and Vannevar Bush’s Memex.” Journal of the American Society for Information Science 43, no. 4 (1992): 284–294. Author’s copy at Berkeleyarchive ↗ (Wayback-preserved; JASIS vol. 43 is not in the Archive’s scanned run — checked).
  • Shaw, Ralph R. “The Rapid Selector.” Journal of Documentation 5, no. 3 (December 1949): 164–171. IA: the-rapid-selector-shaw-1949. The operator’s own account of the ERA-built machine; opens by crediting Goldberg’s 1931 patent and Bush’s electronic system.
  • Dommann, Monika. Authors and Apparatus: A Media History of Copyright. Ithaca: Cornell University Press, 2019. IA: authorsapparatus0000domm. Binkley, the documentalists, and the Gentlemen’s Agreement.
  • Gitelman, Lisa. Paper Knowledge: Toward a Media History of Documents. Durham: Duke University Press, 2014. IA: paperknowledge00gite.
  • Binkley, Robert C. “New Tools for Men of Letters.” The Yale Review 24, no. 3 (Spring 1935): 519–537. IA: sim_yale-review_1935-03_24_3. Also his Manual on Methods of Reproducing Research Materials (1936), IA: manualonmethodso00robe, and Selected Papers, IA: selectedpapersof0000maxh.
  • Hirtle, Peter B. “Research, Libraries, and Fair Use: The Gentlemen’s Agreement of 1935.” Journal of the Copyright Society of the USA 53 (2006): 545–601. Cornell eCommonsarchive ↗.
  • “The Gentlemen’s Agreement and the Problem of Copyright.” Journal of Documentary Reproduction 2 (1939): 29 ff. Not on archive.org (IA holds only JDR vol. 3, 1940, as american-documentation_1940-*); the full run (vols. 1–5, 1938–1942) is full-view and CC-BY-NC-4.0 at HathiTrust record 000546380archive ↗, with per-issue scans in the ALA Institutional Repository.

Prepared from the Internet Archive’s scanned collections; leaf numbers (n##) are BookReader page indexes verified by OCR search on 26 July 2026. External URLs verified present in the Wayback Machine. Updated the same day: added §5 (the copyright question and Binkley), the Shaw 1949 paper (newly added to archive.org as the-rapid-selector-shaw-1949), and the JDR/HathiTrust sourcing. Report by Claude for Brewster Kahle.

Posted in Uncategorized | Leave a comment

The Machine, the Law, and the Link: Bush, Binkley, Nelson

Posted on July 26, 2026 by Brewster Kahle
Research report · Internet Archive holdings · 27 July 2026

Vannevar Bush built the machine and ignored the rights. Robert Binkley fought the rights and had no machine. Ted Nelson designed both into one system — and the world shipped something else. One arc, told from the Archive’s own shelves.

These three stories are usually told separately: Bush as the prophet of hypertext, Binkley as a footnote in copyright history, Nelson as the eccentric who named the link. Read together — and our collections now hold every key document — they are one continuous argument about how scholarship gets command of the record, with each man seizing the part of the problem the others let drop.

Thread one · the machine · 1931–1949Bush: retrieval without rights

The memex story is told in full in the companion report; what matters here is what Bush’s machines touched. The Comparator read Navy cipher traffic. The Rapid Selector’s demonstrations ran on scientific abstracts. The memex existed only in magazine pages. At no point did a Bush machine hold enough of anyone’s copyrighted literature for a publisher to notice — the rights problem never arose because the machine never got that far. In Burke’s archival history of the projects, the only “infringement” is patent infringement, and the closest thing to a publisher dispute is Bush’s agent negotiating reprint permission for the essay itself.

The machine thread has a human splice to the next thread: Ralph Shaw, librarian of the U.S. Department of Agriculture — the same library that ran the documentation movement’s article-copying service — who rebuilt Bush’s Selector after the war and described it in 1949 in a paper we added to the Archive this week (the-rapid-selector-shaw-1949). Its opening page performs the whole lineage: credit to Goldberg’s 1931 patent, credit to Bush’s electronic system, used with Bush’s permission “for the public good.”

Thread two · the law · 1935–1940Binkley: rights without a machine

Robert C. Binkley ran the Joint Committee on Materials for Research with the opposite obsession. His 1935 Yale Review essay “New Tools for Men of Letters” (we hold it: sim_yale-review_1935-03_24_3, from p. 519) imagined micro-copying democratizing scholarship — and he understood immediately that the obstacle wasn’t optics but law. Where Bush’s machines never met a publisher, Binkley spent the decade negotiating with them: the 1935 Gentlemen’s Agreement with the National Association of Book Publishers bought scholars the right to a single copy in place of a loan, at the price of reaffirming the publishers’ exclusive rights. Its text ran in the movement’s own journal — the Journal of Documentary Reproduction, vol. 2 (1939) — whose surviving digital volumes we mirrored into the Archive this week. By 1938 publishers were questioning the agreement’s validity; Dommann’s history (authorsapparatus0000domm) tracks Binkley’s growing disillusionment, his turn against Berne ratification, his insistence that the right to copy is not the right to publish. He died in 1940, at forty-two, mid-fight.

The two threads were braided by the participants themselves, in a title almost too good to be true: in 1947, Vernon D. Tate — MIT’s librarian, and a veteran editor of the documentation movement — published “From Binkley to Bush” in The American Archivist (10:3, pp. 249–257; we hold it: american-archivist_1947-07_10_3, from leaf n26). The movement understood its own succession: Binkley’s cultural program of access was being handed to Bush’s engineering program of retrieval.

The succession Tate named in 1947 took twenty-five more years to produce someone who accepted both inheritances at once.

Thread three · the link · 1960–1993Nelson: both problems in one design

The direct line from Bush to hypertext runs through documents we hold in one volume: Nyce & Kahn (frommemextohyper0000unse_b3u7) reprints Douglas Engelbart’s 1962 letter to Bush alongside Ted Nelson’s 1972 “As We Will Think” — the paper whose title announces itself as the memex’s heir. But Nelson’s real distinction in this arc is that he is the first machine-builder since Binkley’s death to treat the rights problem as part of the engineering. In Literary Machines (three editions on our shelves, e.g. literarymachines00nels) the Xanadu design makes everything published quotable by transclusion — and wires a royalty into the act itself, metered per byte, flowing automatically to the owner of whatever is quoted. It is the Gentlemen’s Agreement rebuilt as system architecture: universal copying, universally licensed, with the publisher’s consent obtained once, structurally, instead of letter by letter.

He was still arguing it at the dawn of the web. In the Archive’s WAIS-era files sit three Nelson papers from 1992–93 — “You Will, Oscar, You Will!: The Implications of Free Quotability and Transpublication” (01Kahle000829), “Publishing Contracts for a Point-and-Click Universe” (01Kahle000846), and “Xanadu Space, 1993” (01Kahle000838) — proposing, on the eve of the web’s takeoff, exactly the rights layer the web would ship without. That these papers survive in this collection at all is its own footnote to the arc: they were in the working files of the people building the next distribution system while Nelson was proposing the licensing for it.

CodaNone of them shipped — and that is the point

Bush’s memex was never built; its component machines worked just well enough to prove the analog path a dead end.

Binkley’s settlement outlived him only as custom; the conflict it deferred returned as Williams & Wilkins, CONTU, and every library-copying fight since — a literature we also hold (williamswilkinsc0001will, reprographycopyr0000hatt, librarycopyright0000duke).

Nelson’s Xanadu never shipped at scale; the web took the link and left the royalty, so the machine finally arrived with the law still unresolved — which is why Binkley’s half of the problem is still being litigated in the era of Bush’s machine.

The dovetail, in one line: Bush built without asking; Binkley asked without building; Nelson designed the asking into the building; the world built without the design. Any library that scans and lends today is living in the space those three outcomes left open.

ApparatusSources — all verified in our collections

Leaf numbers (n##) are BookReader page indexes verified by OCR search, 26–27 July 2026. External URLs verified present in the Wayback Machine (fresh captures triggered for the HathiTrust and ALAIR pages that lacked them). Report by Claude for Brewster Kahle.

Posted in Uncategorized | Leave a comment

Good-by to Gutenberg? Nope—Shrill Publisher Cries about the Xerox Machine Were Wrong

Posted on July 20, 2026 by Brewster Kahle

[written by claude.ai leveraging the new archive.org Supreme Court collection]

Eight years before Jack Valenti called the VCR a “Boston strangler,” America’s publishers told the Supreme Court that the office copier would extinguish the printed word. A field guide to the doom prophecies of Williams & Wilkins Co. v. United States (1973–75) — verbatim from the briefs.

“I say to you that the VCR is to the American film producer and the American public as the Boston strangler is to the woman home alone.”

— Jack Valenti, president, Motion Picture Association of America, testifying to the House Judiciary Committee, April 12, 1982

That line is famous because it was so wrong: within a few years, home video was the single biggest revenue source the film industry had ever known. But Valenti’s Boston strangler had an older cousin. Eight years earlier, in the first case to ask whether a library’s photocopying counted as fair use, the nation’s publishers and authors filed a wall of amicus briefs at the Supreme Court warning that the Xerox machine would destroy scholarly publishing, the free press, and the marketplace of ideas.

The case was Williams & Wilkins Co. v. United States. A Baltimore medical publisher sued the National Institutes of Health and the National Library of Medicine for making single photocopies of journal articles for researchers. The publisher lost — but barely: the U.S. Court of Claims held the copying to be fair use by a vote of 4–3 (1973), and the Supreme Court affirmed by an equally divided 4–4 Court (1975). Read today, the briefs on the losing side are a museum of incumbent-industry catastrophizing. Here are the exhibits.

The exhibits

Exhibit A — the photocopier is a printing press

“Xerography and other copying techniques have already turned every office mailroom into a publishing house.”

Quoting a 1966 Newsweek article literally titled “Good-by to Gutenberg,” the brief argued a Xerox is a “plateless printing press” — so a library’s single-copy service was really mass publishing. (American Society for Testing & Materials et al., amicus brief, 1974; counsel Robert B. Washburn.)

Exhibit B — the journals will simply die

“Periodicals and journals are neither immortal nor immune from the laws of economics… periodicals — e.g. LIFE, LOOK, Saturday [Evening] Post, etc. — were forced to terminate their existence… it is only a matter of time before the same fate overtakes some of these periodicals.”

If magazines that big could fold, the argument ran, photocopying would finish off the scientific and technical journals next. (The Authors League of America, amicus brief, 1974; counsel Irwin Karp.)

Exhibit C — one copy replaces all copies

“‘Sharing’ is a euphemism which in library terms means that by the systematic use of photocopies one published copy of a work can take the place of many.”

The core economic claim: every photocopy is a subscription that will never be sold. (Association of American Publishers & Association of American University Presses, amicus brief, 1974; counsel Charles H. Lieb.)

Exhibit D — a censorship apocalypse

“…the public may well be the loser… because government functionaries, rather than independent copyrightees, will determine what may or may not be published.”

Kill the independent journals, the brief warned, and government publications fill the vacuum — recasting a fair-use ruling as a First Amendment threat. (Magazine Publishers Association, amicus brief, 1974; counsel Alfred H. Wasserstrom.)

Exhibit E — the slippery slope to the end of publishing

“…it would not be difficult to foresee a time when but one copy of a poem or short story or even a novel need be published and be readily made available to all free of charge.”

If a science article, then why not a poem, a story, a whole novel — one copy for everyone, forever. (Associated Councils of the Arts, amicus brief, 1974; counsel Howard M. Squadron.)

Exhibit F — death of the marketplace of ideas

“It further will disrupt and be destructive of the economic relationships on which this nation’s distinctive competitive marketplace of ideas depends.”

The largest possible stakes, for the smallest possible act: one library, one copy, one researcher. (Information Industry Association, amicus brief, 1974; counsel Paul G. Zurkowski.)

The verdict of hindsight

Scholarly and scientific publishing did not collapse. It became one of the most profitable industries in the world — operating margins widely reported in the 30–40% range — and journal subscription prices exploded over the following decades, the library “serials crisis” that is the precise opposite of death-by-photocopier. The Government’s own brief had already pointed out, in real time, that the plaintiff’s own subscriptions were rising: Medicine went from 2,864 to 5,444 over the decade at issue, and Gastroenterology from 4,132 to 7,006.

The Xerox turned out to be a rounding error next to the pricing power the publishers themselves would later wield. “The Xerox will kill journals” aged about as well as “the VCR is the Boston strangler” — and for the same reason. Both were copying technologies an incumbent industry swore would destroy it; both industries proceeded to monetize the technology and thrive. The publishers even got the compensation mechanism they wanted: the Copyright Clearance Center, founded in 1978, turned photocopying into a licensing revenue stream.

The one they got right

To be fair, not every prophecy missed. The American Chemical Society’s brief looked past the copier entirely:

“…the printed journal may someday be obsolete to be replaced by instant around-the-world dissemination of information through video display tubes.”

— American Chemical Society, amicus brief, 1974

That one came true. It just wasn’t the office copier that did it — and when the network did arrive, it made the journals richer, not extinct. The lesson of the copying-machine panics isn’t that new technology is harmless. It’s that the industries most sure a machine will destroy them are usually the ones about to make a fortune from it.

Sources. All amicus quotations are transcribed from the digitized briefs in Williams & Wilkins Co. v. United States, 487 F.2d 1345 (Ct. Cl. 1973), aff’d by an equally divided Court, 420 U.S. 376 (1975) — Internet Archive item micro_IA40385001_1623 (U.S. Supreme Court records & briefs); OCR lightly corrected, bracketed text supplied. “Good-by to Gutenberg,” Newsweek, Jan. 24, 1966, pp. 85–88 (as quoted in the ASTM brief). Valenti testimony: Home Recording of Copyrighted Works, House Judiciary Subcommittee hearings, 97th Cong. (Apr. 12, 1982); the technology reached the Court in Sony Corp. of America v. Universal City Studios, 464 U.S. 417 (1984). Publishing margins / “serials crisis”: RELX/Elsevier reporting; see Stephen Buranyi, The Guardian, June 27, 2017. A full case guide (briefs, the 4–4 deadlock, counsel) is attached to the Internet Archive item as a PDF.

Posted in Uncategorized | Leave a comment

Generative AI is a New Drug: Report on My First ‘Trip’

Posted on February 1, 2026 by Brewster Kahle

I now see how it can be addictive: A mindblowing experience and feeling like I see the world a bit differently. A feeling of being super smart and powerful. Getting lured in with free samples then everything costs. How it grabs my attention when using it, and then when I am not using it, I yearn to get back in.

Is it productive to think of generative AI as a new drug?

I have been thinking of a project in the decentralized web arena and trying to find someone to hire or inspire to try it out. Then a former Archivist gave a workshop at the Internet Archive showing how Slack programmers use Claude Code and it seemed too good to be true. So I thought I would try it. I dove in with a $20/month Claude Code subscription and started making astounding progress in an hour or so. It came up with a plan and started to execute it; we swerved, changed, debugged. I soon ran out of tokens and I needed to pay for more, this time $100/month– ok, lets try it.    Point 1: first bit is free, then the cost escalates.

It flattered me at every step– “Good idea” “That will make a much better UX experience” etc.  I was super good at this, apparently.  It said so.  All the time.   

For 3 years, I have used generative AI tools like ChatGPT and Google AI summeries for one-shot help on spreadsheet formulas, command lines, helpful research, occasional entertaining  poems or pictures. But yesterday I thought I would take out Claude code to see if I could prototype a project I have been dreaming of:  a simple downloadable self-hosted web server that could be used without any of the hassles and costs of domain name registration, https certificate registration, hosting costs.  A web server for a decentralized world.  It is a project that would take a full time person maybe a year to get right and propagated, and that person is hard to find and expensive. Someone called the project “Onion Press.”  Could I vibe code it?

I am a computer programmer but out of date. This tool magically could configure all the newest packages, knew how to take suggestions really well and iterated and fixed its own bugs.  It was like having an impossibly-fast programming guru right there, always smiling.

My mind was blown watching a machine that could take vague suggestions and render them in code– and fast! It is surreal to watch the machine go through iterations debugging the setup– reading logs from multiple components watching things have issues and taking corrective action. It knows much more about these components than I do, and acts successfully. Mindblowing. I am trying to understand the implications of such a new capability being out in the world.

I felt super powerful– maybe now I could make my dreams come true. I even asked chatGPT for a logo for the project with a brief description, and out popped a nifty logo and related icons. In seconds.

After yesterday’s binge, I have been thinking of improvements all day.   I can not wait to get back to the keyboard. It all seems too good to be true. Longing for another session of feeling empowered. And when I could not get back in, I had a feeling of loss, like missing an imaginary friend:

My description here could be someone describing a first psychedelic drug experience, complete with an urge to tell people of my experience.

Then it occurred to me:  Generative AI is a New Drug.   And it doesn’t spill.  Yet.

Posted in Uncategorized | Comments Off on Generative AI is a New Drug: Report on My First ‘Trip’

Ethics Of Digital Librarianship (1992)

Posted on March 23, 2024 by Brewster Kahle

[This essay was included in every Wide Area Information Servers (WAIS) distribution as a way to instruct server operators (later to be called webmasters) on how to deal with log files. Software distributions can be found https://archive.org/details/freeWAIS-sf-1.0

I am proud of this essay. It seems prescient -brewster]

Ethics of Digital Librarianship
Brewster Kahle
Thinking Machines
February 1992

“As digital librarian, you should serve and protect each patron as if she is your only employer.”


As more of us become involved in serving information electronically to other users, we so-called “digital librarians” must become conscious of our ethical responsibilities to protect the privacy of our the users being served. Since computers are being used by many more people to find answers from diverse information sources, we librarians that operate these servers are coming exposed to the exact questions and interests of people we do not know. This information has power, a power that can be abused and thereby thwart the usefulness of the tools we promote. In this essay, I will use the Wide Area Information Server system as an example of a system of digital librarians to show what information is collected and used. With this example, I hope to illustrate some of the dangers and help list some of the rules of etiquette for this emerging class of information providers.


The Wide Area Information Server (WAIS) system is an electronic publishing system that allows end-users to ask questions of remote information sources. The system encourages people to ask questions in natural language so that the server system can try its best to find appropriate documents. Therefore the operator of the server can collect the questions, and importantly, collect what documents the users thought were worth looking at. This combines to portray exact interests of the users. While the identity of the user is not trivial to determine since only the machine that the query came from is accessible from the server logs, as personal computers become networked, the identity of the machine will approximate the identity of the user.


On the positive side, this means that the server operator (the “digital librarian”) can use that data to refine the database and the search techniques used in the system. On the negative side, this is exposing many remote operators to private information that may not be consciously given by the users.


This surrender of information is not new to librarians; and the responsibility is taken very seriously by the professionals in the field. Through training in library schools and by an intuitive sense of ethics, reference librarians do not betray their patron’s interests to others that are curious or devious. This ethical code is not coded in law as it is with psychiatrists, so these records can be extracted through subpoena, but this level of demand is usually required to pry the information from librarians. From the patron’s point of view, having a librarian know what she is interested in can be a great value because the librarian can help select and route useful information in the future.


The same type of information is available to the digital librarians of the WAIS system. I operate the directory of servers in the WAIS system, and as such, I know what users are requesting access to what type of servers. I know, for instance, every time Mitch Kapor uses the system, and what he asks for (he specifically allowed me to include his name here). At this point this is not a problem since few servers are of a personal nature yet, but as the system grows to include entertainment, employment, health and other servers, it is easy to imagine the types of information that will be accessible through operating such a server. Furthermore, I know when particular users are at their machines, and therefore know where they are and when.


The abuses possible with this information are often not as direct as other offenses, but should not be discounted. People will act differently if they think they are being watched. Most people will try not to look silly or ignorant in public, and therefore might be less willing to try something new, to learn about a subject that they know nothing about. If using a WAIS server feels like raising one’s hand in school, then people will craft their questions more carefully than if it felt more like browsing through a new book. Often people say “I have nothing to hide,” which may be true, but if a stranger approaches on the street and knows quite a bit of personal information, then the innocent will likely take that person more seriously than if a cold stranger approached. Even with nothing to hide, most people feel they should who knows what about them. The personal nature of information access makes distributing collected questions a bit unnerving.


The information collected by the digital librarians have some different characteristics from physical librarians which can make abuse easier and more widespread: more people can be served, these people are often in other organizations, and the digital librarians rarely have personal contact with these users. Therefore, the patrons seem further away and therefore less real as human beings. Since the computer networks that are being used with WAIS span the globe and span company boundaries, the information collected can be useful in knowing what is important to a distant, and possibly competitive group. The lack of human contact can lead to the decay in social relations as has been documented in studies of electronic mail where the language and nature of relations tend to be stripped of grace, etiquette, and often respect [cite Sherry Terkle]. This detached nature of electronic interaction might lead librarians to not respect their patrons’ interests where they would if they knew them personally.


On the other hand, the information collected from patrons can be very useful to the digital librarian to refine and enhance the server. An example of this is a reporter at a financial newspaper. She is in the business of collecting information from corporate contacts, finding the trends in that information, throwing out the proprietary details, and selling it back to that same population. If the reporter published too many details, then her contacts would not be forthcoming the next time, and if she sanitized the information to the point of uselessness, similarly, her contacts would not invest the time. Therefore, it is precisely the interaction with the users that builds the information that is sold. This example shows another facet, and that is value that the contacts invest in the reporter for their own benefit. The digital librarian is a less extreme case, but still she is being invested and entrusted with what the users want, and if this information is misused or not used, then the users will not be as well served as could be. Thus, the users will want to be able to be served better by the librarian through feedback on services rendered.


While there are some technological mechanisms to obscure the identity of the patron, such as encryption and redirection, hopefully these will only be used in extreme cases. Encryption can be used to protect packets in transmission and also be used to sign packets so that they can not be forged [cite Whitfield Diffie]. This can be useful in a system where the transport media is insecure, such as radio transmission. Redirection is a server forwarding technique that would concentrate all the requests from one trusted host so that the individual requesters are more difficult to determine. Combinations of these techniques have been contemplated to provably obscure requesters while still providing accountability for charges, but hopefully these techniques will not be the norm if most server operators will act in good faith towards their patrons.


To try to list a code of ethics for this field is difficult since the technology keeps changing, but I will offer a principle that can be used to test a code. As digital librarian, you should serve and protect each patron as if she is your only employer. Therefore each patron should be served and protected individually. In terms of WAIS, I feel it is safe to suggest:

  • Don’t give away user logs except for scholarly use. Consider sanitizing the records before any transfer is undertaken.
  • Take the job of information serving seriously. This means to provide a consistent, reliable service and represent the service provided accurately.
  • Count on wide use of the information served, for good uses and bad, so be proud of the information and the collection.
  • Completeness is important. Users learn as much from a question that has no answer as from the ones with answers. This requires a complete and up-to-date collection.
  • Assume that the patron will not know the your affiliations, and therefore do not tempt patrons to use a service they would regret if they new more about you.
  • Respect your patrons. The opinion that users are “rocks with arms”, as said by a colleague years ago, will not lead you to become a very helpful digital librarian.

In conclusion, the rewards from being a digital librarian are numerous and can be evident from notes from users from remote countries and companies. This electronic publishing revolution allows anyone with a personal computer and a modem to be a publisher will have far reaching effects on the structure of our society. Being a good digital librarian is a concrete way to create a future we all want to live in.

Posted in Uncategorized | Comments Off on Ethics Of Digital Librarianship (1992)

My eBike Economics

Posted on February 7, 2023 by Brewster Kahle

I use an eBike to commute 2.5 miles to work most days. It is faster than driving and parking, and much cheaper than Uber/Lyft.

A few details:

  • Uber/Lyft costs $12 each way, so $5/mile
  • Uber/Lyft takes 20 minutes: 10 minutes to arrive, and 10 minute drive
  • Driving+parking is about 15-20 minutes, depends
  • Driven by my wife costs too many brownie points
  • eBike takes 12 minutes (20mph max), no parking hassle (park inside)
  • eBike cost $1800 + $300 helmet, and wore out ebike in about 6 years
  • eBike costs about 3cents/mile in electricity, so 10 cents each way (I think)
  • Bike would be better, but hills… I just didn’t do it.
  • I had one bike accident, so now wear face-saving helmet and leather jacket.

So $2100 for eBike = 80 round trips on lyft/uber, or about 4 months.

Faster (12minutes vs 20minutes), cheaper, fun, more dangerous.

Been ebiking to work for over 13 years: I wrote a paper 13 years ago about how it was going then.

Posted in Uncategorized | 2 Comments

Grace Lurton Miller 1928-2023

Posted on January 14, 2023 by Brewster Kahle

My dear aunt. Her poem, 1961.

Posted in Uncategorized | Comments Off on Grace Lurton Miller 1928-2023

New account on Mastodon

Posted on November 24, 2022 by Brewster Kahle

The Internet Archive set up a Mastodon instance, and I created an account on it. Feels like a fresh new day. @brewsterkahle@mastodon.archive.org

Mastodon

Posted in Uncategorized | 2 Comments

Pythonistas: Up for quick hack to test Dedup’ing 78rpm records using document clustering?

Posted on October 2, 2022 by Brewster Kahle

I think this could be a 1 day exploration at least to figure out if it will work, but it is beyond my python ability.

Idea: OCR the labels of our 78rpm records, then take an image of a new 78rpm record and list the ones that are close to it. I would think this could be done with a search engine, or it could be with document vectors (gensim). On mac yesterday I got the images and a lead from trusty stackexchange:

pip3 install internetarchive
brew install tesseract
#get the images of the labels (there are 350k of them, but can test with 1000)
ia search "collection:georgeblood" --itemlist | head -1000 | parallel -j10 'ia download {} --no-directories --format="Item Image"'
ls -1 *.jpg | parallel 'tesseract {} {}'

# then something like gensim e.g. https://stackoverflow.com/questions/42781292/doc2vec-get-most-similar-documents


Anyone up for helping test this theory? Again, I am thinking this is a one day hack. If it works, then it will take tuning and such, and the Archive, I would hope, could sponsor that phase.

We have many duplicates in the collection already, so testing this could be easy.

Posted in Uncategorized | 1 Comment

Greenhouse that desalinates its own water: A Desalinating Greenhouse

Posted on September 4, 2022 by Brewster Kahle

An idea. Combine two things to solve a big problem: Solar Water Still and Greenhouse– greenhouse that runs on salt water.

A “solar water still” is often a tent that has salt water at the base and captures the evaporated water and drains it into a bucket.   Solar stills are “the simplest device that are used to obtain freshwater using solar energy as the sole energy supply”.

What if that tent were also a greenhouse, and the fresh water was used to water the plants?  The plants could be in a raised bed that is above the pool of salt water that was under the full floor. This inexpensive construction would be decidedly low tech– low energy inputs, and low maintenance, but we would be growing plants using saltwater.

Then the Desalinating Greenhouse would be a hothouse that grew plants in the freshwater humid warm air and watered with the evaporated water. There would no complicated reverse-osmosis desalinization system or electric energy to drive it– just use salt water to grow freshwater vegetables.

Combine a Solar Water Still:

Solar Water Still for Desalinization

with a Greenhouse:

Generic Greenhouse that could be used as a large solar water still as well

Then we are using the sun to create the freshwater for the plants out of the saltwater. The salt water needs to be supplied and saltier water needs to be removed, but this is a simple pump if there is nearby salt water.

A Solar Still generates .06 gallons to .09 gallons (1/2 to 3/4 lbs) of water per day per square foot, and the peak water use in a greenhouse is 0.3 to 0.4 gallons of water per day. So we need maybe 2-3 times more salt water pool space than growing bed. This does not seem unusual in raised bed greenhouses.

This could be done anywhere there is salt water.  Say coastal regions, on islands that are notoriously short of fresh water, and floated out in the open ocean (living the seasteading dream 🙂 ).   You would need to pump the water to the greenhouse and return the more salty water, but this is low-tech.

Seastead dream

Maybe we could grow fish in the salt water and get some aquaculture going.  (This is not “aquaponics” since that uses the nitrogen from the fish to fertilize the plants, but since these are salt water fix, we can not pour that water on the plants.  A freshwater pool with fish could be a fun add-on if things get mature.)

I bet this whole thing has been tried, as almost everything has, but maybe the coming crisis of fresh water will propel development.   There is a paper about a similar system but seems more complicated than it needs to be:

Complicated combination proposal— I believe we can do better

Another proposal that is a bit less complicated:

Another proposed concept

But I think we can do better. Any ideas, anyone interested in trying?

Posted in Food | 4 Comments