Technological Capabilities: The Algorithm as Editor
What happens when an algorithm selects your news? In 2018, Facebook released a mobile product, called “Today In,” which uses an algorithm to select articles from various news outlets and then provides Facebook users with a daily feed of local news and information. As of May 2020, “Today In” was available in more than 6,000 communities. Using proprietary data provided by Facebook in 2019, as well as more recent samplings, researchers at the UNC Hussman School of Journalism and Media examined the timeliness, relevance and type of local news stories available in North Carolina communities.
Hurricane Dorian made landfall on the Outer Banks of North Carolina on Sept. 5, 2019, causing flooding and trapping hundreds of people on Ocracoke Island. Almost 200 miles inland in the university community of Chapel Hill, subscribers to Facebook’s “Today In Chapel Hill” found two stories about the hurricane in their daily local news feed. One was a day-old story from a local radio station: “Chapel Hill Preparing Ahead of Impact from Hurricane Dorian, Future Hurricanes,” which had also been featured the previous day, Sept. 4. The second, from a Fox-affiliated television station in Greensboro, 50 miles to the west of Chapel Hill, was three days old, “Raleigh-Durham Airport monitoring Hurricane Dorian for potential impact.” There was no news on the progress of the hurricane.
During the first week of April 2020 – a week after the governor of North Carolina had issued a stay-at-home order – “Today In Chapel Hill” featured several stories related to the coronavirus pandemic. Those stories included useful information about what to do if sheltering in place with a domestic abuser, suspension of weekly yard waste pick-up, and a request for food donations for the county emergency operations. But there were no articles that placed into context regional trends in the greater Raleigh-Durham metro area, which includes Chapel Hill. There were also a number of stories from outside the Chapel Hill community – about the International Space Station, a foal found on the Outer Banks, the arrest of boxing promoter Floyd Mayweather’s daughter in Houston, and a story datelined Conroe, Texas, about a woodcarver making canes for veterans.
Facebook launched “Today In” in 2018, based on research that found users of the social media platform wanted more local news.238 The free mobile product, available only on smartphones and tablets, draws content from more than 1,200 news publishers every week. In May 2020, it was available in some 6,000 towns and cities239 - including more than 225 North Carolina communities – and, according to Facebook, 1.6 million people used the feature, which typically posts five top stories a day. The news feed also includes announcements from local governments and school systems.
Facebook does not produce the news on “Today In.” Instead, the algorithm that powers the mobile product selects from content that is produced mostly by newspapers and television and radio stations. If publishers want their articles posted on “Today In,” they must register so Facebook can identify them as legitimate news operations. But with the demise of more than 2,100 local newspapers over the past 15 years, there is a dearth of local news in many communities. For a town that has lost its local newspaper, such as Chapel Hill, the algorithm that selects the news often produces quirky results. Users can feel as if they are in a time warp (as was the case with Hurricane Dorian), or scratch their heads as to why feature stories with out-of-state datelines are considered local news. Even in major metro areas such as Raleigh-Durham, where there are multiple news outlets, including regional television stations and newspapers, the algorithm can still fail to pick up on major news stories and prize-winning investigative series produced by those outlets. “It’s still a work in progress,” says Josh Mabry, Facebook’s local news partnerships lead.240
More Americans get news on social media sites than from print newspapers, and Facebook is the dominant social media site for news, with about 43 percent of Americans getting their news from there.241 “Today In” is one of several major efforts by Facebook to address the rise of news deserts, including a pledge in 2019 to spend more than $300 million over three years to foster partnerships with local news organizations.242 In 2019, the social media company also launched “Facebook News,” which highlights the top national stories of the day selected both by an algorithm and a team of editors and curators employed by Facebook.243
The University of North Carolina is one of four universities provided proprietary data from Facebook in 2019, consisting of 314,000 links to news stories in the “Today In” news feed for the month of February. While researchers at other institutions analyzed national trends,244 UNC researchers focused on analyzing what sort of local news was available to residents in 15 metro areas in North Carolina where “Today In” data was then available.245 In addition, the researchers updated and supplemented their findings by sampling the “Today In” news feeds for Raleigh, Durham and Chapel Hill in September 2019 and April 2020.246
Facebook estimates there is not enough local news (minimum five stories a day) to launch “Today In” in about one in three communities.247 But even in areas where there is enough news, the algorithm often shapes a news diet that is very different from what is offered by a traditional news outlet, such as a newspaper or television station. The algorithm for “Today In” selects news items posted on Facebook based on a variety of factors, which, according to Facebook, includes the timeliness of the article and local relevance, which is, in part, a measure of how often an article is shared in a certain area.
UNC analysis found that the algorithm’s reliance on the number of shares in a certain time period can determine how timely the article is when it is posted, the type of information it conveys and the source (whether it is from television stations or newspapers). Here are some of the major findings:
- Many of the articles on the “Today In” news feed were two or more days old. During the last week in September 2019, for example, more than half of the stories on the “Today In” news feeds for Raleigh, Durham and Chapel Hill were two to four days old.
- Almost half of the stories posted in North Carolina communities in February and September of 2019 concerned either crime or human interest. The crime stories tended to deal with on-going cases or single acts of violence. Typical human interest and crime stories included, “She thought she rescued a kitten. It turned out to be a bobcat” and “Durham police investigating after man found shot dead in parking lot.”
- Because crime and human interest dominated, there were far fewer big-picture stories covering such topics as education, health, politics, infrastructure, economic development or the environment – all topics that the Federal Communications Commission has identified as “critical information needs” for residents of any community.248
- There was a disconnect between what was considered major news that appeared on the front page of newspapers and what appeared on “Today In.” Almost half of the articles on local newspaper front pages during February 2019 analyzed by UNC dealt with political, education or health issues, compared with 13 percent on the “Today In” sites. (Table 1) For example, on Feb. 22, 2019, The News and Observer ran a front page story about the state election board voting to hold a special election in North Carolina’s Ninth Congressional District because of election fraud in 2018, a story with major local and statewide implications. The algorithm didn’t pick up that story, but instead posted a story on “Today in Raleigh” and “Today in Greensboro” about a man in Raleigh renting a hotel room to a prostitute who was a minor. Greensboro is in a different media market than Raleigh, and 75 miles away. (See Table 2 for percent of stories from newspapers on “Today In.”)
- In large metro areas, with multiple media outlets, there are more stories from regional television stations than newspapers. This is likely because television stations have much larger social media followings than newspapers, and, as a result, their stories get shared more often. WRAL in Raleigh, for example, has more than 600,000 Facebook followers compared with 115,000 for The News and Observer.249 Almost 90 percent of the stories in September posted on the news feeds for Raleigh and Durham came from the local television stations. This compares with only 50 percent in Chapel Hill.
- Outside the major metro areas, most of the stories on “Today In” came from newspapers. Overall, slightly more than half of the stories came from newspapers, while a third came from television stations. But even in mid-sized markets that did not have a major regional television news operation, “Today In” tended to select feature stories, not the major news stories that made the front page of local newspapers. For example, stories in The Fayetteville Observer in February 2019 about downtown parking, officials seeking additional tax revenue and food stamp benefits were not posted, while feature stories about a church rebuilding after Hurricane Mathew and another about the Tuskegee Airmen of World War II were.
Researchers at the University of Minnesota and Duke University had similar insights when they analyzed the massive trove of proprietary data on the 400 “Today In” news feeds active in February 2019. To quickly and efficiently analyze the data, they devised their own algorithm that used headlines on the 314,000 links provided by Facebook to determine the type of story – such as sports, emergencies, obituaries, or schools. They were able to classify about half of the links (149,000) into 10 broad categories.250 Eight of those categories (such as emergencies and public safety, education, and health) were identified as serving a “critical information need.”251
Using their own algorithm to sort the stories, the researchers concluded that 60 percent – or 89,000 stories – delivered critical information and news. However, that conclusion was heavily skewed by the number of articles about emergencies and public safety, which includes crime stories. Almost half of the stories classified as delivering critical information were in the emergencies and public safety category, and it was the second most popular category, accounting for 28 percent of total stories. Articles on other critical topics – such as education and politics – fared more poorly, with education stories accounting for only 9 percent and political stories for only 2 percent of the total. Sports stories were the most popular stories in February 2019, accounting for almost a third – more even than emergencies and crime.
Unlike the UNC researchers, the researchers at Duke and the University of Minnesota did not click through to the links to determine timeliness, source or local relevance. Also, they did not compare which major stories were featured on the front pages of local newspapers that were not picked up by “Today In.”252 However, both studies found that the most popular stories on “Today In” are those involving breaking news (emergencies and crime) or human interest and sports.
Mabry, Facebook’s local news partnerships lead, points out, “People can also find local news and community information in other areas of Facebook – [including] publishers they are connected to in News Feed – or products such as Local Alerts, which provides critical information to communities directly from local governments and first responders.” Facebook’s COVID-19 Information Center, for example, has daily updates on the progression of cases in their region, information from local authorities and a feature called Community Help that allows users to request or offer to assist neighbors, as well as donate to relief efforts.
Both research studies, however, concluded that Facebook users who relied exclusively on the “Today In” feature for local news coverage would most likely miss important local governmental, political, economic and educational stories that are written and produced by other news outlets. “Facebook [is] not covering the news,” Jennifer Parker, editor and publisher of the Cross Roads News in Decatur, Georgia, told the Medill News Leaders Project in September 2019. “They’re not covering local government, they’re not covering our local schools.”253 Instead, Facebook is relying on an algorithm that picks news based on what engages users and prompts sharing. Assuming a journalist attends a local planning board meeting and writes a story about it – not always a certainty given the number of local newspapers that have vanished – that type of story is not typically shared as widely on social media as crime or human interest stories.
In June 2020, Facebook announced it would discontinue “Today In” as a separate product, and, instead, would make the local news feed available through Facebook News, which features national news drawn from some of the country’s leading news organizations. In contrast to the “Today In” feature, Facebook News doesn’t rely solely on artificial intelligence to select the articles. It also employs a team of about 30 curators to exercise editorial judgment, and choose the most important stories.254
“When we started talking to news organizations about building Facebook News earlier this year, they emphasized that original reporting is more expensive to produce and better recognized by seasoned journalists than by algorithms. So to help reward this kind of work, we formed a curation team [with editorial independence] to manage the “Today’s Stories” section of Facebook News,” Campbell Brown, Facebook vice president for global news partnerships, said in announcing the launch.255 In addition, some major news organizations with content on Facebook News – including The New York Times, Wired and News Corp. – “are being paid for their content … so we can have access to more of their content.”256
However, Facebook currently has no plans to supplement the selection of local news articles, formerly on “Today In” and now on Facebook News, by hiring additional curators who can identify important stories that might have been overlooked by the algorithm. Nor does Facebook plan to pay the vast majority of local news organizations for their stories, contending that the local news feed will drive users to the originating news sites. However, many of the links UNC researchers tested were behind paywalls and required users to subscribe in order to read the entire story. This most likely discourages users from remaining on the local news organizations’ sites.
“As we add more local news to Facebook News, we will continue testing, updating and improving the algorithm,” said Mabry. “We will continue adding more tools for people to control their news experience such as specifying their location for more local news and tying their paid subscriptions [to news outlets] to Facebook News."
“Today In” was designed to address the desires of Facebook users to see more local news on the social media platform. But Facebook is confronting two issues as it attempts to provide its users with timely and relevant local news, based on UNC analysis. First, the quantity of local news in many markets has fallen off dramatically in recent years with the closure of hundreds of local newspapers and the layoffs of thousands of local journalists. As a result, feature stories from outside the community, and even outside the state, appeared frequently on the local news feeds in North Carolina towns and cities, and half were two or more days old.
There is also a quality issue. Unlike with Facebook News, there is no editor exercising judgment about which local story is more important – a story about a special congressional election versus a story about a tawdry murder. Left to its own devices, without human intervention, the algorithm that selects local news stories is doing exactly what it was designed to do – choosing stories based on a programmed set of criteria that weighs popularity with users, as measured by the number of shares. Therefore, “When you’re looking for actual, original local reporting that fills a critical information need, it’s still newspapers that are, by far, the primary source,” says Philip Napoli, professor of public policy at Duke University and author of an exhaustive study in 2018 that looks at the local news available in 100 mid-sized cities in the country.257
Researchers Bill Arthur, Justin Kavlie (Ph.D. ’21), and Jeremiah Murphy (M.A. ’19) compiled data and provided analysis for this article.