CMU Celebrates Picturephone’s 50th Anniversary By Recreating Historic First Call

City of Pittsburgh, Alcoa Join In Recognizing Legacy of Video Conferencing

On June 30, 1970, AT&T’s Mod II Picturephone was publicly demonstrated for the first time in Pittsburgh, with then-Pittsburgh Mayor Pete Flaherty speaking "face-to-face" with the former chair of Alcoa, John Harper. Thus began commercial video calling service.To celebrate the event's 50th anniversary, Carnegie Mellon University's School of Computer Science and University Libraries collaborated on June 30 to recreate the call. Pittsburgh Mayor William Peduto and Alcoa Chair Michael G. Morris spoke via Zoom, just as their predecessors did a half century ago via the Mod II."I would not have guessed a year ago that this would become such a commonplace way of communicating," Peduto said via Zoom from the City-County building in Downtown Pittsburgh. "The events of the past four months have reinforced just how essential this technology is."Morris said that Alcoa recently held its annual shareholder meeting virtually rather than in-person due to the global pandemic, and praised the convenience of videoconferencing."For the generation that grew up with this, they think the technology is natural," Peduto said. "But imagine fifty years ago. Ten years ago I was still tripping over that phone cord in my mom’s kitchen."Peduto said now he’s video conferencing on a daily basis. He meets frequently with his own staff a few miles away, and with colleagues globally. "Being able to do so seamlessly during a time of a pandemic has made the ability to deliver critical services for the city able to continue."After the call reenactment, a panel including CMU’s Chris Harrison, Andrew Meade McGee and Molly Wright Steenson, considered the legacy and future of video."Besides modern video calls being in color, the end experience is remarkably unchanged," said Harrison, an assistant professor in the Human-Computer Interaction Institute. He recently updated an original set of Mod II Picturephones, which were purchased by CMU University Libraries. The modern computer and screen he installed allow the devices to be used for calls made via Skype, Zoom or FaceTime."That’s true also of the telephone, that the end experience is remarkably unchanged," Harrison said. "And I think that fifty years from now I would not be surprised if we still have telephones and videoconferencing. Those media are successful. What I think will happen is more media will be added."Molly Wright Steenson wondered if holograms will be a part of that future. Steenson, senior associate dean for research in the College of Fine Arts, said she admires the 1970s aesthetic of the Mod II, and pointed out the biggest difference in modern video calling is that it is mobile."You’re no longer calling from place to place, you're calling from a number that floats freely," said Andrew Meade McGee, a CLIR postdoctoral fellow in the History of Science and Computing in the University Libraries. "The hardware of the Mod II is replaced by software you can take anywhere, connecting people regardless of where they are."A video of the event is available on YouTube.

Could Your Computer Please Be More Polite? Thank You

Carnegie Mellon Method Automatically Makes Requests More Polite

In a tense time when a pandemic rages, politicians wrangle for votes and protesters demand racial justice, a little politeness and courtesy go a long way. Now researchers at Carnegie Mellon University have developed an automated method for making communications more polite. Specifically, the method takes nonpolite directives or requests — those that use either impolite or neutral language — and restructures them or adds words to make them more well-mannered. "Send me the data," for instance, might become "Could you please send me the data?" The researchers will present their study on politeness transfer at the Association for Computational Linguistics annual meeting, which will be held virtually beginning July 5. The idea of transferring a style or sentiment from one communication to another — turning negative statements positive, for instance — is something language technologists have been doing for some time. Shrimai Prabhumoye, a Ph.D. student in CMU's Language Technologies Institute (LTI), said performing politeness transfer has long been a goal. "It is extremely relevant for some applications, such as if you want to make your emails or chatbot sound more polite or if you're writing a blog," she said. "But we could never find the right data to perform this task." She and LTI master's students Aman Madaan, Amrith Setlur and Tanmay Parekh solved that problem by generating a dataset of 1.39 million sentences labeled for politeness, which they used for their experiments. The source of these sentences might seem surprising. They were derived from emails exchanged by employees of Enron, a Texas-based energy company that, until its demise in 2001, was better known for corporate fraud and corruption than for social niceties. But half a million corporate emails became public as a result of lawsuits surrounding Enron's fraud scandal and subsequently have been used as a dataset for a variety of research projects. But even with a dataset, the researchers were challenged simply to define politeness. "It's not just about using words such as 'please' and 'thank you,'" Prabhumoye said. Sometimes, it means making language a bit less direct, so that instead of saying "you should do X," the sentence becomes something like "let us do X." And politeness varies from one culture to the next. It's common for native North Americans to use "please" in requests to close friends, but in Arab culture it would be considered awkward, if not rude. For their study, the CMU researchers restricted their work to speakers of North American English in a formal setting. The politeness dataset was analyzed to determine the frequency and distribution of words in the polite and nonpolite sentences. Then the team developed a "tag and generate" pipeline to perform politeness transfers. First, impolite or nonpolite words or phrases are tagged and then a text generator replaces each tagged item. The system takes care not to change the meaning of the sentence. "It's not just about cleaning up swear words," Prabhumoye said of the process. Initially, the system had a tendency to simply add words to sentences, such as "please" or "sorry." If "Please help me" was considered polite, the system considered "Please please please help me" even more polite. But over time the scoring system became more realistic and the changes became subtler. First person singular pronouns, such as I, me and mine, were replaced by first person plural pronouns, such as we, us and our. And rather than position "please" at the beginning of the sentence, the system learned to insert it within the sentence: "Could you please send me the file?" Prabhumoye said the researchers have released their labeled dataset for use by other researchers, hoping to encourage them to further study politeness. In addition to the students, the study's co-authors included several professors from the LTI and the Machine Learning Department — Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov and Alan Black. The Air Force Research Laboratory, Office of Naval Research, National Science Foundation, Apple and NVIDIA supported this research.          

Analysis of Complex Geometric Models Made Simple

Monte Carlo Method Dispenses With Troublesome Meshes

Researchers at Carnegie Mellon University have developed an efficient new way to quickly analyze complex geometric models by borrowing a computational approach that has made photorealistic animated films possible. Rapid improvements in sensor technology have generated vast amounts of new geometric information, from scans of ancient architectural sites to the internal organs of humans. But analyzing that mountain of data, whether it's determining if a building is structurally sound or how oxygen flows through the lungs, has become a computational chokepoint. "The data has become a monster," said Keenan Crane, assistant professor of computer science and robotics. "Suddenly, you have more data than you can possibly analyze — or even care about." Crane and Rohan Sawhney, a Ph.D. student in the Computer Science Department, are taming the monster by using so-called Monte Carlo methods to simulate how particles, heat and other things move through or within a complex shape. The process eliminates the need to painstakingly divide shapes into meshes — collections of small geometric elements that can be computationally analyzed. The researchers will present their method at the SIGGRAPH 2020 Conference on Computer Graphics and Interactive Techniques, which will be held virtually in July. "Building meshes is a minefield of possible errors," said Sawhney, the lead author. "If just one element is distorted, it can throw off the entire computation. Eliminating the need for meshes is pretty huge for a lot of industries." Meshing was also a tough problem for filmmakers trying to create photorealistic animations in the 1990s. Not only was meshing laborious and slow, but the results didn't look natural. Their solution was to add randomness to the process by simulating light rays that could bounce around a scene. The result was beautifully realistic lighting, rather than flat-looking surfaces and blocky shadows. Likewise, Crane and Sawhney have embraced randomness in geometric analysis. They aren't bouncing light rays through structures, but they are using Monte Carlo methods to imagine how particles, fluids or heat randomly interact and move through space. First developed in the 1940s and 1950s for the U.S. nuclear weapons program, Monte Carlo methods are a class of algorithms that use randomness in an ordered way to produce numerical results. Crane and Sawhney’s work revives a little-used "walk on spheres" algorithm that makes it possible to simulate a particle's long, random walk through a space without determining each twist and turn. Instead, they calculate the size of the largest empty space around the particle — in the lung, for instance, that would be the width of a bronchial tube — and make that the diameter of each sphere. The program can then just jump from one random point on each sphere to the next to simulate the random walk. While it might take a day just to build a mesh of a geometric space, the CMU approach allows users to get a rough preview of the solution in just a few seconds. This preview can then be refined by taking more and more random walks. "That means one doesn't have to sit around, waiting for the analysis to be completed to get the final answer," Sawhney said. "Instead, the analysis is incremental, providing engineers with immediate feedback. This translates into more time doing and less time banging one's head against the wall trying to understand why the analysis isn't working." Sawhney and Crane are working with industry partners to expand the kinds of problems that can be solved with their methods. The National Science Foundation, Packard Fellowship, Sloan Foundation, Autodesk, Adobe, Disney and Facebook provided support for this work.

Fifty Years Into "Picturephones," CMU Revamps Original Machine With Modern Twist

SCS Professor and Historian Team Up To Recreate First Commercial Video Call

"Some people in Pittsburgh are seeing voices."That's how AT&T Bell Laboratories advertised its first commercial Picturephone in 1970. The print ad goes on to say that while it would be a lot of work to build a new type of communication network, "Picturephone service is a reality in Pittsburgh, Pennsylvania," and will soon be available in Washington, D.C., and Chicago."While FaceTime, Skype and Zoom are easily accessible these days, when video calling premiered, it was an expensive process reserved for the elite or major public events," said Andrew Meade McGee, a CLIR postdoctoral fellow in the History of Science and Computing in the Carnegie Mellon University Libraries."As soon as the telephone was invented, people thought about how to transmit more than just voices," McGee said. Early 20th century forerunners of fax machines transmitted images over telegraph wires or radio waves, and the 1920s saw the earliest experiments with one-way videocall transmission. The post-World War II period brought efforts to design a reservable video booth system, which connected directly to other distant booths.AT&T invested hundreds of millions of research and development dollars in videophone technology from the 1940s to the 1960s. The Mod I Picturephone appeared at the 1964 World's Fair, but it was the Mod II model that ushered in the first commercial network for video calls."The Mod II was brought out of the laboratory and into the consumer experience as a viable system for everyday use," McGee said. "Just two years before that, the film '2001: A Space Odyssey' showed people the futuristic technology of video calls, and then AT&T took concrete steps to bring it into the world.""It was cutting-edge technology at the time," said Chris Harrison, an assistant professor in the School of Computer Science's Human-Computer Interaction Institute and self-proclaimed amateur historian. "If you wanted to make a phone call in 1970, the analog audio signal would be transmitted on a phone line from one caller to another. But video is a different story."Video requires a much higher bandwidth than audio, and for this, the Mod II required three phone lines. One transmitted voice, just like a regular phone call. "They used some very clever techniques to compress the video onto just two additional analog phone lines," Harrison said.Harrison and McGee both had to see the Mod II for themselves, so in 2019 CMU Libraries purchased at auction a set of two of the Picturephones — originally owned by a Bell Labs engineer — still in their original packaging. "The wooden crate with 1970s foam was really disgusting," McGee said. But the machines inside were impeccable.Harrison said the sleek silver devices looked futuristic when they premiered, and that the exceptional quality is likely the reason they've survived so well. "We both thought it was a really beautiful piece of craftsmanship," he said.To get the machine up and running again, Harrison initially wanted to run video on the original hardware. "There's no public manual that tells us things like how many volts the cathode ray tube screen needs. Rather than risk the precious hardware, we set aside the original electronics and inserted a new telecommunications platform." Harrison installed a computer with a modern screen and camera to simulate the original system.While video calling technology has been changing incrementally for 140 years, McGee said it often takes disruption to see a sudden widespread adoption of technology."Wars and crises are what compel us to make changes to the technology we use. In World War II, there was a large acceleration of tech like rockets, radar and signal processing. Vast sums of money were thrown at seemingly intractable problems. We don't like to move out of our spheres of normalcy, but when we have to, we'll adopt things that previously seemed outlandish," he said.Harrison noted that today's global pandemic, which has forced everything from board meetings to play dates online, has done just that for communications technology. "We now have a perfect storm of people with general exposure to the technology that's been around for a long time, and a sudden society-wide need to solve a problem."As for what's next, McGee said the new ubiquity of video conferencing will inevitably highlight glitches in the technology, but will also inspire visionaries and entrepreneurs imagining what the future could hold. "People will be out there pushing the boundaries of communication. Slightly behind that will be new social norms and a regulatory response by the state and markets to set parameters and policies for video conferencing."Harrison said, based on personal experience, he can imagine a lot of people suffering from so-called "Zoom fatigue."Both researchers agree that looking at where technology came from can offer a good idea of where it's going."Academia is about standing on the shoulders of giants," Harrison said. "You have to deeply understand that history if you're going to push it forward. Fifty years ago the picturephone planted the seeds of what was possible even though it was a commercial failure. Most of what I do in my lab is largely impractical today, but 50 years from now may be mainstream. These things aren't created in vacuums, so you need to have a human-centered vision of computing."CMU University Libraries supports that vision. It houses a cross-campus, interdisciplinary group called HOST@CMU, or the History of Science and Technology at Carnegie Mellon University. And among the Libraries' Special Collections is the Traub-McCorduck Collection, which includes more than 50 calculating machines, letters, books and two Enigma machines. Now the collection also includes the pair of vintage AT&T Picturephones, which were purchased with the support of the Hunt family.CMU Trustee Tod Hunt is the great-grandson of Alfred Hunt, who co-founded the Aluminum Company of America (Alcoa) with six other entrepreneurs in the parlor of his Pittsburgh home. Tod's grandfather and grandmother — Roy and Rachel — donated the money for the construction of CMU's Hunt Library in 1960. And his family's nonprofit, the Hunt Foundation, provided the funds to purchase the vintage AT&T Picturephones that now housed in Hunt Library's Special Collections.That very model of Picturephone, the Mod II, was publicly demonstrated for the first time in Pittsburgh, where the commercial video calling service began. On June 30, 1970, then-Pittsburgh Mayor Pete Flaherty spoke "face-to-face" with the former chairman of Alcoa, John Harper. The two men were about a block away from each other.To celebrate the 50th anniversary of the event — June 30, 2020 — the School of Computer Science and University Libraries are collaborating to recreate the call. Current Pittsburgh Mayor William Peduto and Alcoa Chairman Michael G. Morris will video chat, as their predecessors did a half century ago. Following the call, a panel of CMU scholars including Harrison, McGee and Molly Wright Steenson, senior associate dean for research in the College of Fine Arts, will host a live Q&A, discussing the history and legacy of the Picturephone's launch.The public is invited to join the livestream and can register on the CMU Libraries website.

A Master of Transformations

Bryant Ready for Next Step: Retirement

When Randy Bryant took the helm of Carnegie Mellon University's School of Computer Science in 2004, he quickly realized that SCS, despite its top ranking among computer science schools, had joined its peers in falling a bit behind the research curve. It was a time when Google and Amazon used thousand-machine server farms to perform unimagined feats and develop new computational methods for solving problems. But academics had yet to embrace the power of big data. "We were still thinking in terms of much smaller scale when we looked at data, and not all the things we could do with it," Bryant recalled recently. "So although good research was going on, universities were working at a much smaller scale than industry." Under Bryant's leadership, SCS created a new emphasis on big data — an emphasis that then spread across the country. It was just one of many transformations in the school and discipline that bore Bryant's stamp during his 36-year CMU career. "As a dean, I feel like I just took what was already a great program with some great vision and helped move it along," said Bryant, who will retire on June 30. "I was in a good place at a good time. I do feel like I was able to help the school move forward by recognizing both our own strengths and future trends and help foster that with the university." Bryant, an MIT alum who spent three years on the Caltech faculty, arrived at Carnegie Mellon in 1984, joining an organization — the Computer Science Department (CSD) — that was smaller and more intimate than today's sprawling SCS. With a Ph.D. program and almost all of its funding coming from a large Department of Defense grant, "it was research all the time," he said. In his case, research meant developing software to help computer chip makers design circuitry for then-new "computers on a chip." Between CSD and the Electrical and Computer Engineering Department, CMU had lots of talent in these areas, and Bryant enjoyed the cross-boundary collaboration. As he worked on these design tools, his interests expanded into the field of formal verification. Finding bugs in hardware designs depended on simulation, and there were limits to how much simulation was possible. So Bryant began exploring formal verification — techniques that could prove a design was correct without doing endless testing. He began collaborating with Ed Clarke, who specialized in the field and would later win the Turing Award for his work on a type of verification called model checking. Meanwhile, SCS launched in 1989 and with it came a new undergraduate computer science degree program. While CSD faculty had once focused almost exclusively on research, now they began to think about how best to prepare undergraduates. Bryant embraced this challenge, partnering with David O'Hallaron to create a new course on computer systems — the one with the course number that's the same as CMU's Zip code, 15-213. They also wrote a textbook, now in its third edition. "That really became a transformative part of my career," Bryant said. "To first create a course at CMU and then create a book we could use to export those ideas to the whole world and develop a community around this book at several hundred universities." Jim Morris, then CSD head, saw administrative potential in Bryant and began coaching him for leadership. When Morris became SCS dean, Bryant became CSD head. And when Morris stepped down as dean in 2004, Bryant stepped up to replace him. As dean, Bryant's vision of "big bets on big data" expanded as he visited companies and heard tales from the Language Technologies Institute about how Google had suddenly dislodged them and the other usual suspects in machine translation competitions. Suddenly, Google was making huge progress in translation by using machine learning to analyze bilingual documents side by side. Unlike previous methods, Google's programs didn't understand language structure. They recognized patterns. "Along with other people in the school, we formulated this theme of big data and made that a center point for SCS," Bryant said. "It turned out the timing was perfect and we had the right collection of people and the right resources to do it." He also promoted the idea at the national level, contributing to a paper that influenced the Obama transition team in 2008. During his decade as dean, SCS also launched two new departments: the Machine Learning Department and the Computational Biology Department. In both cases, Bryant said, the new departments followed a CMU pattern: start small, hire young faculty, create bridges to related departments and build the programs carefully until they're ready to be departments. Though new departments flourished, SCS also faced the so-called dot com bust during Bryant's tenure. Student applications, which had been on the rise, suddenly plummeted as high school students began to worry that no jobs were to be had in computer science, or that the jobs were boring, or that all of the jobs would be overseas. "CMU lived through it because we had a big enough applicant pool that was still strong, even though it was shrinking," Bryant said. But then things turned around. New grads started leaving SCS with tantalizing job offers and applications went off the charts. "I describe that as the Mark Zuckerberg effect, because students went from saying, 'Oh, I don't want to spend my life in a cubicle turning out code' to 'Gee, I think I'm going to sit in my dorm room and write code all night.' Same thing, very different attitude," Bryant said. When he stepped down as dean in 2014, Bryant opted to spend a sabbatical year at the Office of Science and Technology Policy (OSTP) in Washington, D.C. He spent much of that time promoting the National Strategic Computing Initiative — a unified plan to establish a high-performance computing infrastructure. "It wasn't my main research area, but it gave me an opportunity to jump into the middle of a government initiative where a lot of good work had been done and that really just needed someone to keep the ball rolling," he recalled. Back at CMU after his year in DC, Bryant returned to teaching and, inspired by his work on the computing initiative, decided to delve more deeply into parallel computing — one of the subjects he had tackled at the OSTP. Kayvon Fatahalian, who was then teaching the parallel computing course, agreed to have Bryant teach the course with him. "Quite honestly, even though I was the more senior faculty member, I was very much the junior partner and I let him do most of the work. He developed the course," Bryant said. When Fatahalian left CMU two years later, though, Bryant suddenly found himself in charge. "I was quite panicked," he said. "It was really stressful to do it for the first time." Three years later, he's still teaching the course, with Nathan Beckmann as co-instructor, but in a much better position. Now, Bryant is making one more transition: into retirement. Though he anticipates staying around CMU and is thinking about the fourth edition of his textbook with O'Hallaron, he said he plans to make it a real retirement, without the pressure of teaching huge classes. He's got grandkids to watch grow, travel plans — including a possible trip to Nepal — and music to enjoy as a member of the Bach Choir of Pittsburgh. "One of the things my father used to say is, you should change your job every six years," Bryant said. "And, in some ways, I've done that at CMU because I started as an assistant professor and then became a full professor and then a department head and then as a dean and now I'm post-dean. So I've fundamentally changed the nature of what I was doing every six years or so. "And I've managed to do it without actually changing employers." SCS has established a fund to honor Randy's commitment to the school. The fund's income will support SCS graduate fellowships.

Two CMU AI Projects Addressing COVID-19 Win Funding

SCS Proposals Are Among 26 Awards From C3.ai Digital Transformation Institute.

Researchers in the School of Computer Science who have proposed using artificial intelligence to mitigate the COVID-19 pandemic are among the first recipients of funding from the newly created C3.ai Digital Transformation Institute (C3.ai DTI). Ziv Bar-Joseph, FORE Systems Professor in the Computational Biology and Machine Learning departments, will head a multi-university effort to model how the SARS-CoV-2 virus responsible for COVID-19 affects the lungs. These models would be used to identify and develop combinations of drugs that hold the most promise for combating the virus. Rayid Ghani, Distinguished Career Professor in the Machine Learning Department and the Heinz College of Information Systems and Public Policy, will collaborate with researchers at CMU and University of Chicago to study AI and COVID-19 policy. Specifically, they'll develop methods and tools to ensure that the AI and machine learning technologies developed to fight COVID-19 do not further the inequities that disadvantage racial minorities and other vulnerable populations when they're used to inform policy decisions. The two projects are among 26 approved by C3.ai DTI, a research consortium established in March by CMU and other leading research universities and national laboratories, including C3.ai and Microsoft. The consortium aims to use AI to speed the pace of digital transformation in business, government and society. Nine universities now belong to the consortium. "The enthusiastic response among scientists and researchers coupled with the diverse, high-quality and compelling proposals we've received suggests that we have the potential to alter the course of this global pandemic," said Thomas M. Siebel, CEO of C3.ai. "In the face of this crisis, the institute is proud to bring together the best and brightest minds and provide direction and leadership to support objective analysis and AI-based, data-driven science to mitigate COVID-19." Bar-Joseph's project will receive $275,000 and involve researchers at MIT and the Boston University School of Medicine. By performing experiments in which they infect lung cells they have previously engineered, the team will develop and validate models that reconstruct the infection pathways of SARS-CoV-2 in the lung and identify key proteins and protein combinations that affect viral loads. The most promising of these models will be used to identify drug targets and to find drug combinations that effectively treat infections while minimizing side effects. "Beyond the immediate impact on the COVID-19 pandemic, the computational and experimental methods for identifying potential drugs and drug targets that will be developed as part of this project will have a long and lasting impact on society's ability to quickly and effectively respond to future global health crises," Bar-Joseph said. Ghani, whose project will receive $250,000, noted that racial minorities and the economically disadvantaged already bear the brunt of the pandemic. He and his collaborators will look for ways to use AI to understand, reduce and mitigate such disparities, especially when policymakers use tools such as those for epidemic forecasting, resource allocation and contact tracing. "I think of our work as an insurance policy against all AI tools that are being developed to support policymakers as part of this effort," Ghani said. "There's a very real risk that if these tools are not developed properly and, more importantly, applied properly, we could make disparities worse."

New Study Explores User Comfort With Privacy Assistants

Jessica Colnago believes that in the future, walking down the street will be a little weird. "You know how every time you enter a website, it says: 'We use cookies. Do you consent?' Imagine that same thing walking down the street, but for a light pole, or a surveillance camera or an energy sensor on a house," Colnago said. Colnago, a Ph.D. student in the Institute for Software Research's Societal Computing program, works with a research team developing personalized privacy assistants (PPAs) — technologies that help people make privacy decisions about devices around them. Without PPAs, "… it's going to be unbearable to live in a world with internet of things devices everywhere giving you notice and asking for consent," Colnago said. In a new study presented at the Conference on Human Factors in Computing Systems (CHI 2020) in May, Colnago and her co-authors from ISR, CyLab and the Heinz College, outlined their efforts to discover how much autonomy people feel comfortable giving PPAs. (You can watch a video of their presentation on YouTube.) "We found that people are definitely interested in having some sort of assistance like that provided by a PPA, but what that assistance looks like varies," Colnago said. The team conducted 17 interviews to gauge participants' reactions to three increasingly autonomous versions of PPAs. The first would simply notify users that devices were near them. A majority of participants reacted positively to this version, while a few viewed noted it would fuel their anxiety. Among the people who indicated they would like to receive such notifications, the majority noted that they would ideally also want to have some control over the data collected about them, rather than just being told about something they couldn't control. The researchers presented the study participants with a second version of a PPA, which would know users' personal preferences on privacy and use that information to make recommendations. A majority of participants also reacted positively to this version, though some of them would rather have the recommendations presented to them based on authoritative sources rather than their personal preferences. The last PPA was the most autonomous: the PPA would leave the user out of the decision-making process entirely and make privacy decisions for them based on their preferences. Reception was mixed. "I would consider owning such an appliance," one participant said. "I don't like to be fully controlled by a device, you know?" another noted. "These interviews told us that there is no single version of a PPA that everyone would be comfortable with," Colnago said. "What we develop needs to include an array of features users can choose from to fit their individual needs and comfort levels." Moving forward, the team aims to develop a system they can actually test with users to determine how they'd react in a more realistic situation. "We gained important insights from these 17 participants, but the scenarios we gave them were all hypothetical," Colnago said. "We need to measure how people would actually behave." Other authors on the study included Yuanyuan Feng, a, ISR post-doctoral student; Tharangini Palanivel, a master's student in the Heinz College; Sarah Pearman, a Ph.D. student in societal computing; Megan Ung, an undergraduate computer science student; Alessandro Acquisti, a professor in the Heinz College; Lorrie Cranor, director of CyLab and professor in both ISR and Engineering and Public Policy; and Norman Sadeh, professor in ISR and PPA project director.

Schwartz Named Head of Computational Biology Department

Martial Hebert, dean of Carnegie Mellon University's School of Computer Science, has named Russell Schwartz the new head of the Computational Biology Department, effective July 1. Schwartz, a professor in both the Computational Biology Department (CBD) and the Department of Biological Sciences, will succeed Robert F. Murphy, who founded CBD and is stepping down after 13 years as its leader. "Russell has a stellar record as an educator and researcher, with balanced expertise in both computer science and biology," Hebert said. "I am sure that he will build on the foundation laid by Bob Murphy to bring the Computational Biology Department into its next phase." Schwartz, who joined the CMU faculty in 2002 and also has appointments in both the Computer Science and the Machine Learning Departments, works broadly on models and simulations of biological systems, including work in computational genomics, phylogenetics, population genetics and biophysics. In recent years, his lab has largely focused on computational cancer biology, especially the development of computational methods to understand how populations of cells that make up a tumor evolve as the cancer develops. Such work is important for better predicting which cancers or precancerous lesions are likely to be aggressive and how they might respond to potential treatments. Schwartz previously co-directed CBD's Ph.D. program, offered jointly with the University of Pittsburgh. He currently serves as a co-director of the joint University of Pittsburgh-CMU Medical Scientist Training Program and as vice chair of Carnegie Mellon's University Education Council. "The future of science will unite the foundational sciences with computer and data sciences to address the most critical issues facing society," said Rebecca W. Doerge, Glen de Vries Dean of the Mellon College of Science. "Russell's dedication to interdisciplinary work in his lab and in our classrooms make him an excellent choice for moving computational biology forward at Carnegie Mellon." As part of the School of Computer Science, CBD develops and uses cutting-edge computational approaches to drive scientific research, enabling discoveries that could not be made with traditional means. In addition to Ph.D. and master's programs, the department offers an undergraduate major in computational biology, as well as a new master's program in automated science. "As Bob Murphy steps down as head of the Computational Biology Department after 13 years, we salute his leadership in implementing his vision of a world-class department with a unique array of breakthrough research and educational programs," Hebert said. Murphy, the Ray and Stephanie Lane Professor of Computational Biology, will continue to pursue research in both experimental and computational cell biology, with a particular emphasis on developing fully automated methods to understand the subcellular locations of proteins and how they change during development or disease.

Zazzle Uses MHCI Capstone Ideas To Create New Products

Online Marketplace Benefits From Collaboration With Student Teams

Wuyang Wang knew she got lucky last year. Then a student in the Human-Computer Interaction Institute's (HCII) Master of Human-Computer Interaction program (MHCI), she was assigned to a capstone project team for Zazzle, a platform that brings people together to design, make, sell, purchase or even share products online. That assignment proved a bit of good fortune. The capstone project is a highlight of the MHCI program, giving groups of students an opportunity to work with clients on real-world solutions to their needs and wants. As Wang would learn, Zazzle is a perennial sponsor and an unusually approachable and collaborative client. Notably, Zazzle values the student input so highly that it incorporates many of the concepts the MHCI students develop into its product and service offerings. "A lot of our products coming this year are focused on content and collaboration," said Lonny Chu, senior director of user experience within Zazzle's product group. "The genesis of all that was previous capstone work." This year marks the fifth that Zazzle has sponsored an MHCI capstone project, he noted, and each student to date has made important contributions to the company's brand identity and products. "I love every one of them," Chu said. "They're all terrific individuals." Skip Shelly, associate teaching professor in the HCII and director of the MHCI program, said the capstone project gives students an opportunity to apply the HCI design methods they learned in the previous semester to an intensive, seven-month engagement with a sponsor. "The students use a variety of qualitative and quantitative research methods to develop useful insights, and to reframe and find problems worth solving," he said. "Students engage in iterative research, design and validation cycles that mirror the rhythms of industry." Sponsors such as Zazzle benefit by seeing prototypes of their ideas, often incorporating advanced technologies such as machine learning or conversational agents, said Jessica Vogt, HCII engagement manager. "For our students, the capstone yields real project experiences and deep content for their portfolios that distinguishes them in the industry," Vogt said. "That's an important reason students choose the MHCI program."

CMU Method Makes More Data Available for Training Self-Driving Cars

Additional Data Boosts Accuracy of Tracking Other Cars, Pedestrians

For safety's sake, a self-driving car must accurately track the movement of pedestrians, bicycles and other vehicles around it. Training those tracking systems may now be more effective thanks to a new method developed at Carnegie Mellon University. Generally speaking, the more road and traffic data available for training tracking systems, the better the results. And the CMU researchers have found a way to unlock a mountain of autonomous driving data for this purpose. "Our method is much more robust than previous methods because we can train on much larger datasets," said Himangi Mittal, a research intern working with David Held, assistant professor in CMU's Robotics Institute. Most autonomous vehicles navigate primarily based on a sensor called a lidar, a laser device that generates 3D information about the world surrounding the car. This 3D information isn't images, but a cloud of points. One way the vehicle makes sense of this data is by using a technique known as scene flow. This involves calculating the speed and trajectory of each 3D point. Groups of points moving together are interpreted via scene flow as vehicles, pedestrians or other moving objects. In the past, state-of-the-art methods for training such a system have required the use of labeled datasets — sensor data that has been annotated to track each 3D point over time. Manually labeling these datasets is laborious and expensive, so, not surprisingly, little labeled data exists. As a result, scene flow training is instead often performed with simulated data, which is less effective, and then fine-tuned with the small amount of labeled real-world data that exists. Mittal, Held and robotics Ph.D. student Brian Okorn took a different approach, using unlabeled data to perform scene flow training. Because unlabeled data is relatively easy to generate by mounting a lidar on a car and driving around, there's no shortage of it. The key to their approach was to develop a way for the system to detect its own errors in scene flow. At each instant, the system tries to predict where each 3D point is going and how fast it's moving. In the next instant, it measures the distance between the point's predicted location and the actual location of the point nearest that predicted location. This distance forms one type of error to be minimized. The system then reverses the process, starting with the predicted point location and working backward to map back to where the point originated. At this point, it measures the distance between the predicted position and the actual origination point, and the resulting distance forms the second type of error. The system then works to correct those errors. "It turns out that to eliminate both of those errors, the system actually needs to learn to do the right thing, without ever being told what the right thing is," Held said. As convoluted as that might sound, Okorn found that it worked well. The researchers calculated that scene flow accuracy using a training set of synthetic data was only 25%. When the synthetic data was fine-tuned with a small amount of real-world labeled data, the accuracy increased to 31%. When they added a large amount of unlabeled data to train the system using their approach, scene flow accuracy jumped to 46%. The research team will present their method at the Computer Vision and Pattern Recognition (CVPR) conference, which will be held virtually June 14–19. The CMU Argo AI Center for Autonomous Vehicle Research supported this research, with additional support from a NASA Space Technology Research Fellowship.

Grenell Joins IPS as a Senior Fellow

Richard Grenell, the former U.S. ambassador to Germany and former acting director of National Intelligence, has joined Carnegie Mellon University's Institute for Politics and Strategy (IPS) as a senior fellow. IPS is a joint initiative of the university's Dietrich College of Humanities and Social Sciences, the College of Engineering, and the School of Computer Science. Grenell brings a decade of experience in diplomacy and international relations to Carnegie Mellon, where he will engage with students and faculty. He spent eight years as the U.S. spokesperson at the United Nations before assuming the role of U.S. ambassador to Germany. "Grenell joins the Institute for Politics and Strategy at a critical juncture in our nation's history," said IPS Director and Taube Professor Kiron Skinner. "Having served a decade at the State Department, representing the United States at the highest levels at the United Nations and later as ambassador to Germany, I could not think of anyone more qualified to help the nation think through 21st century diplomacy." In 2004, Grenell was appointed an alternate representative to the UN Security Council with full voting rights and privileges. His service as UN spokesperson coincided with worldwide upheaval following the September 11 terrorist attacks. He crafted communications strategies related to the war on terror, Middle East conflict, nuclear proliferation and the security of Israel. Grenell holds a master's degree in public administration from Harvard University's John F. Kennedy School of Government and a bachelor's degree in government and public administration from Evangel University. Read the full story on the CMU News website. Learn more about the decision to appoint Grenell in Skinner's statement on the issue.

Self-Driving Cars That Recognize Free Space Can Better Detect Objects

What a Perception System Doesn't See Can Help It Understand What It Sees

It's important that self-driving cars quickly detect other cars or pedestrians sharing the road. Researchers at Carnegie Mellon University have shown that they can significantly improve detection accuracy by helping the vehicle also recognize what it doesn't see. Empty space, that is. The very fact that objects in your sight may obscure your view of things that lie further ahead is blindingly obvious to people. But Peiyun Hu, a Ph.D. student in CMU's Robotics Institute, said that's not how self-driving cars typically reason about objects around them. Rather, they use 3D data from lidar to represent objects as a point cloud and then try to match those point clouds to a library of 3D representations of objects. The problem, Hu said, is that the 3D data from the vehicle's lidar isn't really 3D — the sensor can't see the occluded parts of an object, and current algorithms don't reason about such occlusions. "Perception systems need to know their unknowns," Hu observed. Hu's work enables a self-driving car's perception systems to consider visibility as it reasons about what its sensors are seeing. In fact, reasoning about visibility is already used when companies build digital maps. "Map-building fundamentally reasons about what's empty space and what's occupied," said Deva Ramanan, an associate professor of robotics and director of the CMU Argo AI Center for Autonomous Vehicle Research. "But that doesn't always occur for live, on-the-fly processing of obstacles moving at traffic speeds." In research to be presented at the Computer Vision and Pattern Recognition (CVPR) conference, which will be held virtually June 13–19, Hu and his colleagues borrow techniques from map-making to help the system reason about visibility when trying to recognize objects. When tested against a standard benchmark, the CMU method outperformed the previous top-performing technique, improving detection by 10.7% for cars, 5.3% for pedestrians, 7.4% for trucks, 18.4% for buses and 16.7% for trailers. One reason previous systems may not have taken visibility into account is a concern about computation time. But Hu said his team found that was not a problem: their method takes just 24 milliseconds to run. (For comparison, each sweep of the lidar is 100 milliseconds.) In addition to Hu and Ramanan, the research team included Jason Ziglar of Argo AI and David Held, assistant professor of robotics. The Argo AI Center supported this research.

Three SCS Faculty Members Named Wimmer Fellows

Three School of Computer Science faculty members — Michael Hilton, Stephanie Rosenthal and Joshua Sunshine — have been named 2020-21 Wimmer Faculty Fellows by the university's Eberly Center for Teaching Excellence and Educational Innovation. The fellowships, sponsored by the Wimmer Family Foundation, are designed for junior faculty members who seek to enhance their teaching by designing or redesigning a course, creating innovative new teaching materials, or exploring a new pedagogical approach. Fellows work in close collaboration with Eberly Center colleagues and receive a stipend. Hilton, an assistant teaching professor in the Institute for Software Research (ISR), and Sunshine, senior research fellow and director of ISR's undergraduate research program, are designing a new course called Crafting Software that will teach programming to students outside computer science or computer engineering who need programming skills as part of their research. Rosenthal, assistant teaching professor in the Computer Science Department, is redesigning Autonomous Agents, a senior-level, artificial intelligence course that she developed in which students deploy and test an AI system in the form of an autonomous greenhouse.

Perception of Privacy vs. Utility Key to Widespread Contact-Tracing App Adoption

CMU Researchers Find Users Prefer Centralized System

Contact-tracing could help curb the spread of COVID-19. While the process can be performed manually, researchers have suggested that digital contact tracing using cell phones could be a more accurate and scalable approach. But its effectiveness relies heavily on a large installation rate — and that may depend on how people weigh the app's utility versus its privacy risks. Researchers at Carnegie Mellon University examined user preferences on six different app designs after explaining the risks and benefits of each option — including whether the user's data was stored on a centralized (government) server or decentralized server run by the app's developer. "Surprisingly, contrary to the assumptions of some previous work, we found that the majority of people in our sample preferred to install apps that use a centralized server for contact tracing," said Tianshi Li, a doctoral student in the School of Computer Science's Human-Computer Interaction Institute (HCII.) Contact-tracing apps may need to collect a lot of sensitive health and personal information, including where you've been, who you've been interacting with and if you've been diagnosed. "The problem is that decentralized solutions are not risk free," Li said. "We found that people are more willing to allow centralized authorities to access information than to allow a decentralized server to potentially offer loopholes to tech-savvy users who could infer the identity of diagnosed users." The largest cluster of people, 32% of the sample, preferred centralized versus decentralized servers. The second largest cluster, 25%, were the most privacy-conscious and disagreed with almost all app designs. Another design aspect included in the survey is location data sharing. Researchers found that a majority of the sample preferred to install apps that share diagnosed users' recent locations in public places to show infection hotspots. "People are generally very sensitive about location data, but in this specific case, users wanted useful information, beyond even direct exposure notice, so they feel more in control of the situation and can make their own decisions about how to reduce risk," Li said. The researchers offer several suggestions for an app design that may achieve a high adoption rate in the U.S. First, servers should be centralized, although Li underscored the importance of handling data in a secure and privacy-preserving manner, and verifying the users' identities during sign up to avoid malicious users identifying diagnosed users. Also, a one-size-fits-all solution has its challenges. Researchers found that at the state level, political leaning influenced design preference. Li said a combination of manual and digital contact tracing may be necessary. The researchers' second suggestion is to provide users with information about infection hotspots, which may nudge them to install the app. Location data collection should be opt-in, and the app should offer multilevel options when it requests that data to accommodate different user preferences. Finally, researchers said these apps should be transparent about the risks of disclosing personal information to both governments and tech-savvy users. "I think this is the very first step to understand the design space," Li said. "Challenges are obvious, such as keeping a centralized server secure from hackers. But now is the time to think about design, before investments are made and apps make their way onto people's phones." Currently, Apple and Google are only offering APIs for decentralized contact-tracing apps. Researchers believe that similar APIs may also be needed to support the implementation of centralized contact-tracing apps that follow the best security and privacy practices. The paper "Decentralized Is Not Risk-Free: Understanding Public Perceptions of Privacy-Utility Trade-Offs in COVID-19 Contact-Tracing" is available now on arXiv. The research team also includes Jason Hong of the HCII and both CMU's Electrical and Computer Engineering Department and CyLab Security and Privacy Institute; Cori Faklaris and Laura Dabbish of the HCII; Yuvraj Agarwa, from CMU's Institute for Software Research; and Junrui Yang and Jennifer King from Stanford University.

DARPA Names Sherry to ISAT Study Group

The Defense Advanced Research Projects Agency (DARPA) has named Justine Sherry, assistant professor in the Computer Science Department, to the Information Science and Technology (ISAT) Study Group. Her three-year term begins this summer. DARPA established the 30-member ISAT Study Group in 1987 to support its technology offices and provide continuing independent assessment of the state of advanced information science and technology as it relates to the U.S. Department of Defense. Sherry's research centers around networks, focusing on making them faster, more reliable, secure, fair and equitable. Recently, her group showed that a new congestion-control algorithm could treat internet requests unfairly by placing preference on some requests, inadvertently slowing others down. "It's hard not to notice DARPA's crucial role in the development of networked technologies everywhere, from support for privacy tools like Tor to the invention of the internet itself," Sherry said. "I am honored and very excited to be part of the conversation about the next generation of transformative technologies supported by DARPA."

Carnegie Mellon Tool Automatically Turns Math Into Pictures

Visualizations Poised To Enrich Teaching, Scientific Communication

Some people look at an equation and see a bunch of numbers and symbols; others see beauty. Thanks to a new tool created at Carnegie Mellon University, anyone can now translate the abstractions of mathematics into beautiful and instructive illustrations.The tool enables users to create diagrams simply by typing an ordinary mathematical expression and letting the software do the drawing. Unlike a graphing calculator, these expressions aren't limited to basic functions, but can be complex relationships from any area of mathematics.The researchers named it Penrose after the noted mathematician and physicist Roger Penrose, who is famous for using diagrams and other drawings to communicate complicated mathematical and scientific ideas."Some mathematicians have a talent for drawing beautiful diagrams by hand, but they vanish as soon as the chalkboard is erased," said Keenan Crane, an assistant professor of computer science and robotics. "We want to make this expressive power available to anyone."Diagrams are often underused in technical communication, since producing high-quality, digital illustrations is beyond the skill of many researchers and requires a lot of tedious work.Penrose addresses these challenges by enabling diagram-drawing experts to encode how they would do it in the system. Other users can then access this capability using familiar mathematical language, leaving the computer to do most of the grunt work. The researchers will present Penrose at the SIGGRAPH 2020 Conference on Computer Graphics and Interactive Techniques, which will be held virtually this July due to the COVID-19 pandemic."We started off by asking, 'How do people translate mathematical ideas into pictures in their head?'" said Katherine Ye, a Ph.D. student in the Computer Science Department. "The secret sauce of our system is to empower people to easily 'explain' this translation process to the computer, so the computer can do all the hard work of actually making the picture."Once the computer learns how the user wants to see mathematical objects visualized — a vector represented by a little arrow, for instance, or a point represented as a dot — it uses these rules to draw several candidate diagrams. The user can then select and edit the diagrams they want from a gallery of possibilities.The research team developed a special programming language for this purpose that mathematicians should have no trouble learning, Crane said."Mathematicians can get very picky about notation," he explained. "We let them define whatever notation they want, so they can express themselves naturally."An interdisciplinary team developed Penrose. In addition to Ye and Crane, the team included Nimo Ni and Jenna Wise, both Ph.D. students in CMU's Institute for Software Research (ISR); Jonathan Aldrich, a professor in ISR; Joshua Sunshine, an ISR senior research fellow; cognitive science undergraduate Max Krieger; and Dor Ma'ayan, a former master's student at the Technion-Israel Institute of Technology."Our vision is to be able to dust off an old math textbook from the library, drop it into the computer and get a beautifully illustrated book — that way more people understand," Crane said, noting that Penrose is a first step toward this goal.The National Science Foundation, Defense Advanced Research Projects Agency, the Sloan Foundation, Microsoft Research and the Packard Foundation supported this research.