服務故事

Traditionally, one may think that Electrical Engineering and Music are two disciplines that are far apart from each other. People typically believe it is more likely for an electrical engineer to take on music than for musicians to take on engineering. Professor Yi-Wen Liu of the Department of Electrical Engineering, National Tsing Hua University (NTHU) and Professor Yi-Hsuan Yang of AL Lab are examples of technical people taking on music. However, when I did my postdoc at Stanford in 1993, many postdocs and researchers had a prior degree in music. One example was Xavier Serra. Xavier is one of the founders of Vocaloid technology. He grew up playing cello and guitar, has a master's degree in music, moved to the study of audio signal processing for music applications for his PHD, and is now working on Computational models for the discovery of the world's music.

The Fantasy of Music Signal Analysis

What does music have to do with AI? Think about the movie "2001: A Space Odyssey." There was an AI computer that sang a nursery rhyme and music digitally composed by Max Matthew. For as long as the term AI existed, it had a close connection to music. AI made it possible to complete unfinished music creations by the likes of Bach, Mozart, Schubert, and Puccini.

My SCREAM Lab at National Cheng Kung University (NCKU) has made some small but meaningful progress on music score analysis in recent years. We explored using supervised learning to segment music scores and determine performing styles.

It was interesting to arrive at different results when we used Mozart and Haydn's music as two different training data sets. The way they interpreted music differently reflected the differences in their music styles. We were fascinated by this finding. While there are many more and deeper studies to pursue, as of this year, this research finding has been incorporated into the techniques used by our virtual symphony orchestra.

Out of an Obsession with Sound and Music

The initial idea of an augmented reality (AR) musical experience is represented by the virtual singer "Hatsune Miku," but we wanted to take it a step further. Sometimes, musicians need accompaniment for performances, and other times, composers need an orchestra to help debut their work. Hiring an orchestra can be costly. A virtual orchestra fills that void.

We started by having one cellist in our first performance. We are now able to present a string quartet. We expect to add woodwind and brass and have a full orchestra within a year. This multi-discipline collaboration of composers, instrumental performers, computer scientists, and animation artists. Such collaboration is common abroad; job and innovation opportunities are plentiful.

Programming: A Modern Bridge for Multiple Discipline Studies

In Taiwan, we often confine ourselves to our field of study. That is not the case abroad. Anyone who analyzed Bach and Beethoven would tell you that these great musicians would have made good mathematicians.

Programming language and music are both languages. To me, the former is much easier.

A programming language has its grammar and logic. A programming language has fewer grammar rules than a natural language and music, but the rules must be fully observed. A programming language also has simpler logic. One may combine multiple simple calculations to execute a complex calculation. Programming is the composition of grammar and logic to carry out a job. In programming, there is no room for ambiguity. Well-written and poorly-written code yields the same output. For people well-versed in music and natural languages, picking up programming is typically not difficult. I have students with a psychology background in my master's program. They can pick up enough programming to accomplish a lot within a year.

Since programming is a language, one would think it is better to learn a language sooner rather than later, but I think the difference is minor. For example, kids today start learning English at a younger age than I did and speak better English than I do. However, although I have a heavy accent, my English is adequate for reading technical papers, doing some simple writing, and conversing with foreigners; to me, that is sufficient. If your interest is not in a technical field, you don't need to be an exquisite programmer. If you are a musician or artist, you only need to know enough coding to validate your creation.

Programming skill often empowers you to realize ideas. Picking up such skills before college is useful, but it is not too late to wait till college. Do not stress over it. It is more important to learn the skill for the right purpose, as with learning English. You should learn to use it, not to pass a test.

AI is a progressive field in universities. There are all kinds of classes on AI frameworks and algorithms, and interdisciplinary research topics are common. AI's application in music is an example. The combination of technology, programming, and interdisciplinary knowledge is already powerful. With the addition of AI, one can do more and better.

"To Be or Not To Be."

In Taiwan, we tend to confine ourselves to our domain of studies and our comfort zones. It is a shame that students are good at taking exams. Still, it is not actual practice; music majors must become musicians, hardware designers are inadequate at software, and people who work with their hands feel inhibited from enriching themselves in theoretical subjects. Although we don't have the resources at Junyi as they did when creating Hatsune Miku, we are proud to be able to demonstrate what technology can do for music in our virtual concert on March 12. Moving forward, with the help of technology, we hope to do more with music.

In this fast-paced world, we should not confine ourselves to one career or find our passion and curiosity only in school and work. We should constantly pursue new knowledge and learn new skills. I learned that from many of my peers and my students. Junyi students get more exposure to this notion than others. I trust you all can find your passion if you put your heart and mind to it.

Original article written in Chinese by Alvin Su, English translation by Angela Chin

Photo credit: Anita Hu, Junyi School of Innovation