{"id":57944,"date":"2019-01-13T21:14:11","date_gmt":"2019-01-14T02:14:11","guid":{"rendered":"https:\/\/www.ludwig-van.com\/toronto\/?p=57944"},"modified":"2019-01-13T21:14:11","modified_gmt":"2019-01-14T02:14:11","slug":"feature-how-researchers-at-the-university-of-toronto-aim-to-help-musicians-with-artificial-intelligence","status":"publish","type":"post","link":"https:\/\/www.ludwig-van.com\/toronto\/2019\/01\/13\/feature-how-researchers-at-the-university-of-toronto-aim-to-help-musicians-with-artificial-intelligence\/","title":{"rendered":"FEATURE | How Researchers At The University Of Toronto Aim To Help Musicians With Artificial Intelligence"},"content":{"rendered":"
\"\"
Technology is breathing vital new life into the world of classical music, just as it needs it most.<\/figcaption><\/figure>\n

C<\/span>lassical music and cutting edge technology may not seem, at first glance, to have much to do with each other. In the public imagination, in fact, classical music may seem like the last stronghold of music making without the use of modern tech. But, to see them as two separate \u2014 or even opposing worlds, is a fallacy. They don\u2019t just intertwine; technology is breathing vital new life into the world of classical music just as it needs it most.<\/p>\n

One of the ways that technology and classical music intersect, increasingly, is through enhancing the audience experience. Giddy headlines may declare that technology will save classical music, but the importance of broadening the reach, and potential appeal, of classical music through digital technology can’t be underestimated. The TSO pushes its recordings and videos through a number of channels, including Spotify, Soundcloud, and a YouTube channel.<\/p>\n

Other orchestras and companies are experimenting with a variety of audience enhancement technologies. Teatro Lirico di Cagliari, an opera company based in Sardinia, used Google Glass, a form of smart glasses, in an experiment in 2014. During a production of Puccini, the singers, musicians, and even stagehands posted pictures and live video feeds directly to the company’s social media accounts. Since 2016, the Boston Symphony Orchestra has incorporated a “Conductor Cam” section of seats in their Casual Friday concert series. In the designated section, patrons get a loaner of an iPad that has enriched content like exclusive interviews. Seated behind a large flat screen, the audience members get the same view of the musicians as conductor Andris Nelsons.<\/p>\n

But, the audience experience is just the beginning. Eric Baptiste, the CEO of Society of Composers, Authors and Music Publishers of Canada (SOCAN), has partnered with the U of T\u2019s Department of Computer Science Innovation Lab (DCSIL) on a project designed to benefit Canadian musicians from a different angle \u2014 that of being able to track music played around the world in order to collect and distribute the royalties. It’s ironic that, as the ability to proliferate music across the globe has become easier and easier, artist revenues have dwindled. The project will use AI \u2014 artificial intelligence \u2014 in various ways to hunt down and track music under copyright wherever it is played.<\/p>\n

ARTIFICIAL INTELLIGENCE<\/h3>\n

It\u2019s not the only way that AI has entered the realm of music. Steve Engels is an Associate Professor, Teaching Stream in computer science at the University of Toronto, but he made headlines last year for making music \u2014 or rather, forgetting AI to make music.<\/p>\n

\u201cI was doing AI related to text,\u201d he recalls. The project initially looked at getting AI to recognize text via an algorithm. It was a student\u2019s interest in music that led to the realization they could look at musical compositions the same way. The algorithm uses what it calls tokens, or a basic unit that it considers as it looks for patterns. \u201cAI looks at it note by note,\u201d he says. Essentially, they feed music to AI, and AI learns. \u201cIf you know a composer well enough, you know what they would do next,\u201d he points out. \u201cGive it a large body of music, and it will begin to recognize patterns.\u201d<\/p>\n

\u201cIt has no concept of music theory,\u201d he says. Without a background in music theory, AI bases its compositions entirely on what it has heard. \u201cIt doesn\u2019t do what it hasn\u2019t already seen. It\u2019s all entirely derivative.\u201d Or, almost. There is some room for something like creativity at times. If AI encounters a few notes, and then recognizes multiple options for what happens next in its listening repertoire, it can then improvise a little at the note or structural level of the music. \u201cThe overall piece has variations.\u201d<\/p>\n

The level of variety depends on what it has heard. \u201cAt first, it would focus on a single piece. Then we started to combine these pieces.\u201d<\/p>\n

While the media focused on the novelty of non-human musical composition, the exercise led Engels to consider the implications. \u201cIs this how people do it too?\u201d he wonders. In earlier eras, in particular, where style was uniform across geographic areas, and Western music followed the common practice tonality, musical composition would have been influenced almost entirely by what the composer had previously heard in their life. \u201cThat\u2019s built into the kind of art we do.\u201d<\/p>\n

While AI may already have found its applications in the world of pop music, it probably won\u2019t find its way onto a classical concert stage anytime soon. \u201cThis is entirely going to be derivative,\u201d Engels says. As he explains, AI is good at imitation. It can simulate a specific composer it\u2019s been given, but the form would depend on what it had been fed. \u201cIf you gave it all concertos, it would come up with a concerto.\u201d If you expose it to many different types of compositions, then the form would become more and more haphazard as it tries to incorporate all the different patterns it has learned. \u201cThere\u2019s still that high-level function that we need humans for. We look at it as a compositional assist.\u201d<\/p>\n

A NEW WAY FORWARD<\/h3>\n

As the lines between music and technology blur, the possibilities open up. Willy Wong is an Associate Professor, Biomedical Engineering \/ Computer Engineering, and director of the Engineering Music Performance Minor program, launched in the Fall 2018 semester. It came about as a recognition of the fact that so many engineering students were musicians too.<\/p>\n

Wong points out that the overlap between engineering and classical music is not new. Julian Andreas Kuerti, conductor and son of Anton Kuerti, got an honours degree in engineering and physics from UofT before switching to music. Soprano Isabel Bayrakdarian studied biomedical engineering at the UofT before taking to the stage.<\/p>\n

\u201cMusic is a very important part of their professional practice as engineers,\u201d he notes. \u201cWe\u2019re also pushing out a Music Technology Certificate.\u201d It\u2019s taking the fusion of music and technology to the student level in a new way. \u201cUniversities tend to be compartmentalized,\u201d he says. \u201cUniversity tends to be a much more theoretical space.\u201d Combining the two disciplines takes both in new directions.<\/p>\n

Wong teaches the physics of sound. \u201cI teach the only course in sound in the engineering program,\u201d he says. \u201cWhere does sound come from?\u201d For him, the fields of engineering and music are two angles on the same thing, and his love of music goes back to childhood. \u201cMy entire family is music based,\u201d he says, including kids in lessons, and a wife \u2014 violinist Etsuko Kimura \u2014 who is an Assistant Concertmaster with the TSO.<\/p>\n

It took some time to get the cooperative program running, but he\u2019s excited about the possibilities that will emerge as students go through it. \u201cIt\u2019s a vision here we want to take forward.\u201d<\/p>\n

Jeff Wolpert is an Adjunct Professor: Music Technology & Digital Media, and Director, Music Technology and Digital Media Program, Sound Recording. \u201cI designed this program,\u201d he says. He’s won multiple Juno Awards for Recording Engineer of the Year, and is a working professional along with is teaching duties at UofT, producing records, movie scores, and other projects with some of the big names of the music business.<\/p>\n

Enrollment in the program, focusing on music production, and music as media, had some initial surprises. \u201cIt seemed to open a side door,\u201d he says. Many students were people who had studied music, and then left the business, or even those who\u2019d had an interest, but never gone into the field, with a good contingent of mature students in their 40s and even older. In other words, it has opened up an alternative pathway to the field of music to the traditional performance major, destined for a career on the stage.<\/p>\n

For Wolpert, technology fills in the next step after learning your craft as a musician. \u201cThe production part is making it happen,\u201d he says. He developed the program as the realization of a need he saw. \u201cNumber 1 \u2014 to keep people in music,\u201d he says. \u201cYou had people with music degrees and no visible means of support. They were taught about live performance, but my question is \u2013 is this how they will make a living?\u201d There is less money to be made in performance these days, and as a composer, he sees a mastery of technology as absolutely crucial.<\/p>\n

In his professional practice, he works on music for the movies, what he calls \u201cthe last bastion of classical music.\u201d As he notes, there are only a few centres in the world, such as Los Angeles or New York City, where orchestral musicians can get regular work recording music for movies. Everywhere else, composers rely on electronic renditions. \u201cOne of the things they really have to do is show the movie director what the music is like.\u201d That means being able to produce a high-quality electronic version of your score. \u201cSometimes the quality of the mock up makes or breaks the deal. Most composers I know are really expert in that technology.\u201d<\/p>\n

Once they get the gig, in his experience, he says composers most often spend virtually their entire commission on music production, making their money through the back end via royalties. \u201cIn most cases, you have to deliver the production.\u201d No more paper scores, no more live performance.<\/p>\n

Digital technology can be a composer\u2019s best friend in more ways than one. \u201cThey\u2019re always worried about repeating themselves,\u201d Wolpert says. With technology, a composer can create a palette of sounds to be used for electronic and electroacoustic music making. He mentions that prominent film composers like Thomas Newman and Hans Zimmer will experiment extensively with the technology, looking for inspiration.<\/p>\n

He was part of the team working with composer Jonathan Goldsmith (Take This Waltz, Sharkwater Extinction<\/em>) on one project. \u201cWhat we built for him was something called the Infinity Table. Technically, it\u2019s not an instrument at all.\u201d Instruments can be plugged into the Infinity Table to produce a virtually infinite range of different sounds. \u201cWriting with technology has become a possibility.\u201d<\/p>\n

Part of the push to compose assisted by tech comes from diminishing timelines for commissions, which he says often involve short lead times of three weeks to three months. \u201cIt\u2019s about finding inspiration on that schedule.\u201d<\/p>\n

But, along with the benefits of tech, Wolpert also recognizes the limitations. \u201cI greatly miss the community,\u201d he says of the new, isolated way of producing music. When it comes to his students, he emphasizes that technology can only be as brilliant as the mind wielding it. \u201cJust because you can program a string section, doesn\u2019t mean you know anything about strings. Much of the character of an instrument is in the articulation,\u201d he points out.<\/p>\n

If classical music needs technology nowadays \u2014\u00a0technology still needs humans. For now.<\/p>\n","protected":false},"excerpt":{"rendered":"

Technology is breathing vital new life into the world of classical music, just as it needs it most.<\/p>\n","protected":false},"author":64,"featured_media":57948,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"jetpack_post_was_ever_published":false},"categories":[25164,4967,61],"tags":[25904,25905,9494],"yst_prominent_words":[25867,25903,25870,25944,25940,6767,6715,7555,25871,25869,25942,6616,25868,25866,25941,25627,25872,25943,16035,16034],"acf":[],"jetpack_featured_media_url":"https:\/\/www.ludwig-van.com\/toronto\/wp-content\/uploads\/sites\/2\/2019\/01\/Music-And-Digital-Technology-At-UofT.jpg","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9bakr-f4A","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/posts\/57944"}],"collection":[{"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/users\/64"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/comments?post=57944"}],"version-history":[{"count":7,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/posts\/57944\/revisions"}],"predecessor-version":[{"id":57996,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/posts\/57944\/revisions\/57996"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/media\/57948"}],"wp:attachment":[{"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/media?parent=57944"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/categories?post=57944"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/tags?post=57944"},{"taxonomy":"yst_prominent_words","embeddable":true,"href":"https:\/\/www.ludwig-van.com\/toronto\/wp-json\/wp\/v2\/yst_prominent_words?post=57944"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}