Carnegie Mellon Team Dives Into DARPA Subterranean Challenge

Modularity Will Be Key to Robotic Exploration of Caves, Tunnels, Underground Structures

A team from Carnegie Mellon University will compete in the systems track of the Defense Advanced Research Projects Agency Subterranean Challenge, a multiyear robotics competition with a $2 million prize in which robots will autonomously search tunnels, caves and underground structures. The Carnegie Mellon team, including a key member from Oregon State University, is one of nine teams that will receive up to $4.5 million from DARPA to develop the robotic platforms, sensors and software necessary to accomplish these unprecedented underground missions. The robots will be tasked with rapidly mapping, exploring and exploiting complex underground environments, ranging from spaces so small that humans can only crawl through them to areas big enough to accommodate an all-terrain vehicle. The challenge is designed to provide both warfighters and first responders with the capabilities they need to accomplish a variety of missions in caves, tunnels or urban underground facilities, such as subway stations. "Successfully completing these missions will require multiple robots, including both drones and ground vehicles," said Sebastian Scherer, who will lead the team with Matt Travers, both of CMU's Robotics Institute. "Our team has a wealth of experience in operating robots in mines, enclosed spaces and the wild, and in coordinating the activity of multiple robots." Travers, a systems scientist in the Robotics Institute, said the CMU team also will leverage its expertise in modularity — developing robots that can be rapidly built and reconfigured to adapt to widely varied environments. "We can't be sure that a four-wheeled platform will always be the right robot for every job, so we need to be ready to add wheels or substitute tracks or even legs," Travers explained. "In some environments, small robots might be our only option, while others may demand larger, more robust robots." Scherer, a senior systems scientist, said communications will be a major challenge underground, and that getting robots to work cooperatively to ensure a space is comprehensively mapped is critical. Geoff Hollinger, an assistant professor of mechanical engineering at Oregon State and a CMU robotics alum, has been recruited to the team for his expertise in multirobot systems. CMU's team also includes Howie Choset, the Kavčić-Moura Professor of Computer Science; Sanjiv Singh, research professor of robotics; Anthony Rowe, associate professor of electrical and computer engineering; and a number of undergraduate, graduate and post-doctoral researchers. "Creating robots that can work in subterranean environments will expand the potential application of robots both underground, such as in mines, and inside structures, such as buildings, ships and aircraft," Scherer said. "The constraints robots encounter in these confined spaces are enormous, so we have our work cut out for us." In addition to DARPA funding, the team is receiving support from the Boeing Co. and from Near Earth Autonomy, a spinoff of the Robotics Institute. They are also seeking additional commercial and foundation sponsors. Beginning in the fall of 2019, DARPA will conduct a series of challenges, including one each in man-made tunnels, natural caves and underground structures. A final event in the fall of 2021 will combine all three types of subterranean environments. The DARPA competition takes place in two tracks: the systems track, in which teams such as CMU develop and demonstrate physical systems for live competitions; and a virtual track, in which teams develop software and algorithms to compete in simulated environments. DARPA will award $2 million to the winner of the systems track and $750,000 to the winner of the virtual track. Carnegie Mellon teams won the 2007 DARPA Urban Challenge robot race and placed third at the 2015 DARPA Robotics Challenge for disaster response robots.

Innovators Convene for First ProductCamp Pittsburgh

Carnegie Mellon University and the University of Pittsburgh hosted the first ProductCamp Pittsburgh Saturday, Sept. 22, at the Tepper Quadrangle, where more than 260 attendees from around the region came to learn from nationally recognized experts in product management. ProductCamps, free gatherings trending worldwide, focus on developing and bringing new products to market. They tout an "unconference" format in which participants forgo an agenda and cast votes in real time about what they want to discuss next. The process aims to elicit feedback, a basic tenet of product management. "ProductCamp Pittsburgh provided so many diverse perspectives. I met students, entry-level professionals, senior directors, entrepreneurs and professors from many disciplines all in one short day," said Amy Super, product experience director at Confluence. CMU's Greg Coticchia led the event team, with support from the Integrated Innovation Institute and Pitt's Innovation Institute. Organizers came from CMU and Pitt, as well as Bally Design, Bayer Health Care, pHase2 microtechnologies, Smart Hammer Innovation and Zoll Lifevest. The all-volunteer effort took six months to plan and attracted corporate sponsors. Coticchia is the executive director of the Master of Science in Product Management (MSPM), a joint program between CMU's School of Computer Science and the Tepper School of Business. "Events like ProductCamp Pittsburgh will inspire existing product managers to be better, and those that want to be product managers to get involved and be a part of the community," Coticchia said. Students, including Murtaza Nasikwala, a first-year MBA student, made up one-third of the attendees and were among the 29 volunteers who helped run the event. "Something students think about is customer discovery, or how to involve diverse groups of people in developing new ideas and features... It's not as easy as sitting alone in a room and coming up with a product," Nasikwala said. Three product management thought leaders gave keynote presentations. The morning featured Eric Boduch, a 1996 graduate of CMU's Department of Electrical and Computer Engineering, a contributor to ProductCraft and the Product Love podcast. Boduch and a fellow ECE graduate co-founded the software company Pendo to develop product management tools. The company has grown to 250 employees in five years. The afternoon offered talks by Steve Johnson, founder and CEO of Under10 Consulting; and Jeff Lash, vice president and group director at SiriusDecisions. Johnson entered product development in the 1980s at a time when product managers were self-taught and mentored. Professionals now have access to tools and education. "Product management, this profession of guiding what products we develop, is coming into maturity," Johnson said. Glassdoor ranked product manager ninth among the best jobs in America for 2018. "Many successful product management leaders — especially at startups — are like the new rock stars. People admire them and want to become product managers because of them," Lash said. The skills are needed in Pittsburgh, where the innovation economy is getting noticed by people like Phil Marzolf, an entrepreneur-in-residence at Pitt. "I lived in the Bay Area for 20 years and never once felt goosebumps about being there physically, but I've had that feeling three or four times since moving here," Marzolf said. Attendance at ProductCamp Pittsburgh exceeded expectations for a first-time event. "Pittsburgh continues to become a 'product and innovation town' with the presence of Duolingo, RoBotany, Bellefield Systems and other success stories," Coticchia said. "ProductCamp legitimizes the strength of similar tech-based companies in Pittsburgh, and CMU is at the forefront of educating product leaders with several degrees that emphasize technology and business, such as the new MSPM degree." Coticchia said the demand for product managers has never been greater. "'Product' isn't what's happening just in Silicon Valley or Silicon Alley; it's big and getting bigger in Pittsburgh," he said.

Carnegie Mellon Will Help Build 3D Cellular Map of Human Body

NIH's HuBMAP Promises Deep Insight Into How Different Tissues Function

Carnegie Mellon University computer scientists are helping the National Institutes of Health build a 3D map of the human body showing how tissues differ on a cellular level and providing deep insights into the body's inner workings. The NIH today announced its first set of research funding awards for the Human BioMolecular Atlas Program (HuBMAP). The NIH anticipates spending $54 million on HuBMAP over the next four years, pending available funds, including more than $2 million for the CMU scientists and their colleagues at the University of California Santa Cruz and the Wellcome Sanger Institute in the U.K. "This is really the next step in the Human Genome Project," said Ziv Bar-Joseph, a professor of computational biology and machine learning in Carnegie Mellon's School of Computer Science who will lead the CMU-based center. "Though the genome — the complete set of genes — is present in all cells, it's clearly doing different things in different organs and tissues. The job now is to create a map that shows which genes and proteins are activated in each part of the body." HuBMAP will provide scientists with a deeper understanding of how the tens of trillions of cells that constitute the body are organized in tissues to carry out the daily processes that keep people alive and healthy. The organization, specialization and cooperation of different cells in each tissue have a profound effect on tissue growth, function and aging. The ability to detect subtle changes in the activity of individual immune cells, for instance, might provide an early warning of disease before symptoms are clinically detectable. "We're excited for HuBMAP to start its journey to expand our understanding of the principles of tissue organization," said Dr. James Anderson, director of the NIH Division of Program Coordination, Planning and Strategic Initiatives. "We expect HuBMAP to provide a vital framework for global efforts to comprehensively understand the human body at a biomolecular level." While some of the HuBMAP research groups will generate new data on cell organization and variability for particular tissues or organs, the HuBMAP Computational Tools Center will focus on the computational methods necessary for processing massive amounts of raw data, constructing the high-resolution tissue maps and building an atlas of tissue maps. "Building a high-resolution map of the cells within our bodies is a major challenge for both biologists and computer scientists, and will provide a huge payoff for our understanding of disease and aging," said Andrew Moore, dean of CMU's School of Computer Science. "I can't think of a better team for making HuBMAP a reality than the group Ziv has assembled." Other key CMU participants include Robert F. Murphy, head of the Computational Biology Department; and Carl Kingsford and Jian Ma, both associate professors of computational biology. The Pittsburgh Supercomputing Center, a program of CMU and the University of Pittsburgh, also is among the groups receiving research awards for HuBMAP. The center will have two co-principal investigators. One is Benedict Paten, an assistant professor of biomedical engineering at the University of California, Santa Cruz, home of the UCSC Genome Browser — the most popular browser for genomic data. The other is Sarah Teichmann, head of cellular genetics at Wellcome Sanger Institute and co-chair of the organizing committee for the privately funded Human Cell Atlas, a project that is complementary to the public HuBMAP. HuBMAP is funded by the NIH Common Fund, which addresses emerging scientific opportunities and high-priority challenges for the NIH. The National Heart, Lung and Blood Institute; National Institute of Biomedical Imaging and Bioengineering; and the National Institute of Diabetes and Digestive and Kidney Diseases manage the program along with the Common Fund staff.

Cassell Says #MeToo Movement Could Effect Long-Sought Changes in Tech Industry

Associate Dean Cassell Is Keynote Speaker at Grace Hopper Celebration

Justine Cassell is one of three keynote speakers at this year's Grace Hopper Celebration of Women in Computing Conference, the world's largest gathering of women technologists. Her topic — how social interaction improves robot performance — is at the heart of her research agenda. But she'd really rather discuss a second talk she's giving at the Sept. 26–28 conference in Houston, the one about how the #MeToo movement is helping women recognize systemic sexism within the tech world. "I'm seeing something happen now that I last saw in the 90s," said Cassell, associate dean of technology strategy and impact for the School of Computer Science. It was common then for women in computer science to declare, "I have never experienced sexism of any kind," she said. Yet a common complaint among women, then as now, was that they were often dismissed as difficult. "Because of the #MeToo movement, many women are realizing that such a criticism may not be their fault," she said. "If it's happening to all of us, it can't be just about me." This sexism persists despite efforts to boost the number of women in the field. Virtually half of the first-year students at SCS have been female for the past three years, for instance, and industry leaders have supported programs to boost the number of women in the tech pipeline for years. Yet women continue to be underrepresented in the tech industry generally, and particularly in technology jobs within the industry. And at the very highest levels, women are just a small percentage of executives. "We are leaking women from the top of that tech pipeline," Cassell said. "For the most part, this is not done intentionally, of course," Cassell said. "There are plenty of well-meaning people." But an unconscious bias persists, she maintained, which leads to women too often being evaluated based on their weaknesses, while men are more often evaluated based on their strengths. Women may tend to internalize these criticisms and blame themselves for the failure to progress in the field, or to be elected to positions of power and authority as they become more senior. But the #MeToo movement is helping them realize the problem lies beyond themselves, Cassell said. "It's not you. You may or may not be difficult. But you're not any more difficult than anyone else in the field," she added. This realization is important for effecting change. "Only when a bunch of people see this systemic bias and complain about it do things change," Cassell said. "So you have to make it visible." Her speech is one of a number being presented by prior winners of the Anita Borg Institute Women of Vision Awards. Cassell won the 2008 award for leadership. As usual, Carnegie Mellon will be well-represented at the three-day conference, with 40 SCS graduate and undergraduate students planning to make the trip, said Carol Frieze, director of Women @ SCS and SCS4ALL. Lani Frazier, who earned a master's degree in information technology management at the Heinz College of Information Systems and Public Policy in 2002, is co-chair of the conference, which is co-presented by the AnitaB.org (formerly the Anita Borg Institute) and the Association for Computing Machinery. Distinguished Career Professor of Computer Science Lenore Blum will host a dinner for the SCS women attending the conference. For more information, visit the Grace Hopper website.

Education App Uses Photos To Help People Form Proper Sentences

CMU Design for America Project Assists People With Language Deficits

Students in the Carnegie Mellon University chapter of Design for America (DFA @ CMU) have developed a free mobile app that uses visual cues to help children and adults — particularly those with language difficulties — form meaningful, grammatically correct sentences. The education app, called Sentence Mosaics, uses photos and color-coded parts of speech to prompt users as they construct sentences.The app is useful for elementary school students and for a wide range of children and adults who are on the autism spectrum, or who have developmental disabilities, hearing disorders, stroke-related language deficits, dementia and other problems with language, said Alyse Baker, a certified speech-language pathologist who has honed the system for decades."Alyse has used colored blocks, colored cards and foam tiles attached to a foam board with Velcro as aids, but that's not practical outside the classroom," said Mehar Sawhney, a DFA @ CMU member and a senior majoring in mathematics and physics. "We worked with her to translate her system into an iPad app that can be taken anywhere."Baker developed Sentence Mosaics based on long-standing research and practice within the field of speech-language pathology. She knew that transforming the system into an app would make sense, but found the prospect daunting."When I first looked into developing an app, I was told it would cost $20,000 to $30,000 to write the code," Baker said. "I did not have that much money in my hip pocket, so I knocked on doors for six years until a stroke of luck took me to David Kosbie." Kosbie, an associate teaching professor of computer science at CMU, connected her two years ago with DFA @ CMU, an interdisciplinary group of students who design and engineer solutions for social good.A group of DFA members, including Sawhney, computer science majors Elizabeth Board and Sashank Gogula, and cognitive science major Elizabeth La, worked with Baker to translate physical aids into digital ones. Jennifer Chou, who graduated last spring with a computer science degree, was also a major contributor to the project.Photos are key to the system. Users write sentences that describe actions depicted in photos, preferably ones that include the student or show a scene from their everyday life. Board said the app can import photos from an iPad or iPhone.Baker explained the photos make the subject real to students, which for many is a necessary step before they can write. "Some of my students, if they didn't have a dog of their own, literally could not write a sentence about a dog," she added.The app is designed to be used with a teacher, parent or other coach, who helps by loading photos into the app and designating a string of colored boxes that signal to the student which parts of speech — nouns, pronouns, verbs, adverbs, adjectives, etc. — are necessary to describe the photo."Without the intercession of Dave Kosbie and countless hours of programming by students at Carnegie Mellon, this app would not have become a reality," Baker said.More information about the app is available on the Sentence Mosaics website. The free iOS app can be downloaded from the App Store.

Von Ahn Wins 2018 Lemelson-MIT Prize for Invention

School of Computer Science alumnus Luis von Ahn, who is a consulting professor in the Computer Science Department (CSD) and the co-founder of the popular language-learning platform Duolingo, has won the prestigious 2018 Lemelson-MIT Prize — a $500,000 award that honors mid-career inventors. The prize recognizes Von Ahn for his groundbreaking inventions, including Duolingo, Games With a Purpose and reCAPTCHAs, as well as his commitment to youth mentorship and to improving our world through technological invention. As a CMU graduate student mentored by Manuel and Lenore Blum, he pioneered the idea of "human computation," a concept better known today as crowdsourcing, and invented computer-based games that accomplished tasks, such as describing online photos so they could be searchable. He also was one of several CSD researchers to develop CAPTCHAs, tests designed to discern human web users from bots. After earning his Ph.D. in computer science in 2005 and joining the CSD faculty in 2006, von Ahn sold the image-labeling game to Google. Later, he also sold Google reCAPTCHAs — an innovation that both provides web security and digitizes old books and periodicals. Von Ahn co-founded Duolingo in 2011 with Severin Hacker, who earned his Ph.D. in computer science from Carnegie Mellon in 2014. The free service now has more than 200 million users and offers 81 language courses across 37 languages. Duolingo also created the Duolingo English Test to make it easier and cheaper for students in developing nations to certify their English skills when applying to U.S. universities. His previous honors include a MacArthur Fellowship. He has been named one of the 10 Most Brilliant Scientists by Popular Science Magazine, one of the Top Young Innovators Under 35 by MIT Technology Review and one of the 100 Most Innovative People in Business by Fast Company Magazine. The Lemelson-MIT Program is funded by the Lemelson Foundation and administered by MIT's School of Engineering. The prize was announced on Wednesday, Sept. 12, at EmTech MIT, a conference in Cambridge, Mass., hosted by MIT Technology Review. Von Ahn also was slated to present an acceptance speech.

Beyond Deep Fakes

Transforming Video Content Into Another Video's Style, Automatically

Researchers at Carnegie Mellon University have devised a way to automatically transform the content of one video into the style of another, making it possible to transfer the facial expressions of comedian John Oliver to those of a cartoon character, or to make a daffodil bloom in much the same way a hibiscus would.Because the data-driven method does not require human intervention, it can rapidly transform large amounts of video, making it a boon to movie production. It can also be used to convert black-and-white films to color and to create content for virtual reality experiences."I think there are a lot of stories to be told," said Aayush Bansal, a Ph.D. student in CMU's Robotics Institute. Film production was his primary motivation in helping devise the method, he explained, enabling movies to be produced more quickly and cheaply. "It's a tool for the artist that gives them an initial model that they can then improve," he added.The technology also has the potential to be used for so-called "deep fakes," videos in which a person's image is inserted without permission, making it appear that the person has done or said things that are out of character, Bansal acknowledged."It was an eye opener to all of us in the field that such fakes would be created and have such an impact," he said. "Finding ways to detect them will be important moving forward."Bansal will present the method today at ECCV 2018, the European Conference on Computer Vision, in Munich. His co-authors include Deva Ramanan, CMU associate professor of robotics.Transferring content from one video to the style of another relies on artificial intelligence. In particular, a class of algorithms called generative adversarial networks (GANs) have made it easier for computers to understand how to apply the style of one image to another, particularly when they have not been carefully matched.In a GAN, two models are created: a discriminator that learns to detect what is consistent with the style of one image or video, and a generator that learns how to create images or videos that match a certain style. When the two work competitively — the generator trying to trick the discriminator and the discriminator scoring the effectiveness of the generator — the system eventually learns how content can be transformed into a certain style.A variant, called cycle-GAN, completes the loop, much like translating English speech into Spanish and then the Spanish back into English and then evaluating whether the twice-translated speech still makes sense. Using cycle-GAN to analyze the spatial characteristics of images has proven effective in transforming one image into the style of another.That spatial method still leaves something to be desired for video, with unwanted artifacts and imperfections cropping up in the full cycle of translations. To mitigate the problem, the researchers developed a technique, called Recycle-GAN, that incorporates not only spatial, but temporal information. This additional information, accounting for changes over time, further constrains the process and produces better results.The researchers showed that Recycle-GAN can be used to transform video of Oliver into what appears to be fellow comedian Stephen Colbert and back into Oliver. Or video of John Oliver's face can be transformed into a cartoon character. Recycle-GAN allows not only facial expressions to be copied, but also the movements and cadence of the performance.The effects aren't limited to faces, or even bodies. The researchers demonstrated that video of a blooming flower can be used to manipulate the image of other types of flowers. Or clouds that are crossing the sky rapidly on a windy day can be slowed to give the appearance of calmer weather.Such effects might be useful in developing self-driving cars that can navigate at night or in bad weather, Bansal said. Obtaining video of night scenes or stormy weather in which objects can be identified and labeled can be difficult, he explained. Recycle-GAN, on the other hand, can transform easily obtained and labeled daytime scenes into nighttime or stormy scenes, providing images that can be used to train cars to operate in those conditions.More information and videos are available on the Recycle-Gan website.

Andrew Moore Will Rejoin Google as Cloud AI Lead

Google announced today that Andrew W. Moore, dean of Carnegie Mellon University's School of Computer Science, will rejoin the company in January to lead its Cloud AI efforts. "I am bursting with excitement about this," Moore said. "I have always deeply believed in the power of technology to improve the state of the world, so for me it's a big opportunity to help Google bring useful AI to all the other industry verticals." Moore recently said that he would step down as dean and as professor of computer science and robotics by the end of the year. At Google, Moore will replace Fei Fei Li, who will resume her full-time role as a professor of computer science at Stanford University. In his new role, Moore will be based in Pittsburgh. "We are incredibly fortunate to have Andrew's leadership at this point in our development as we define how we will expand bringing AI and ML (machine learning) technologies and solutions to developers and organizations all over the world," the company said in a Google Cloud Blog post. Moore joined the SCS faculty in 1993 and was recruited by Google in 2006, where he led the creation of Google Pittsburgh. He was Google's vice president of engineering before returning to the university as dean in 2014. "On behalf of the Carnegie Mellon University community, I am excited to convey our support for Andrew Moore as he takes this next step in his career," said Farnam Jahanian, president of Carnegie Mellon. "In addition to advancing CMU's vision for the transformative potential of AI, Andrew is exemplifying the Carnegie Mellon ideal of taking ideas developed at the university and applying them to real-world environments. Pursuing a shared understanding of the vital role of AI in the 21st century will also continue to strengthen our long-standing partnership with Google." As dean, Moore leveraged SCS's pioneering work in artificial intelligence, establishing a CMU AI initiative that now includes more than 200 faculty from across campus. He has been an outspoken proponent of AI, testifying before Congress and consulting regularly with government and industrial leaders. Though AI sometimes is depicted as a possible menace, Moore has contended that AI holds great potential for good and has urged researchers to always consider the societal impact of their work. Jahanian said he will appoint an interim dean and launch a national search for Moore's replacement. "I want to express my deepest thanks to the amazing students, faculty and staff of SCS," Moore said in announcing his resignation on Aug. 28. "This school is extremely strong and remains a thought-leader in computer science and robotics. With the technological changes now facing mankind, the world has never needed a strong SCS more than it does today."