Code Meets Composition: Creating a Visual Soundscape with Creative Coding

Eagle standing on top of a computer with the screen showing a picture of the earth. Planets float in the background of the image.
Eagle standing on top of a computer with the screen showing a picture of the earth. Planets float in the background of the image.
The DKC’s Creators Podcast
Code Meets Composition: Creating a Visual Soundscape with Creative Coding
Loading
/

Transcript 

(Intro music)     

Angela: Welcome to the Digital Creator Podcast, the show where we spotlight students working on cool digital projects. I’m Angela, a consultant at the Digital Knowledge Center, where we empower students to be digital creators. Today’s guest, Sean, composed an original piece of electronic music and wrote code to generate a custom visual that brings the sound to life. 

Sean: I’m Sean Foley. I use he/him pronouns. I’m a computer science major and I will be graduating this year either in the summer or fall. 

Angela: Cool. So, tell us a little bit about your project. 

Sean: Broadly speaking, what I’m doing is writing a piece of music and then using code to create a visualizer for it. 

Angela: Okay, that sounds really cool. So how did you get into – what inspired this project? 

Sean: A short deadline. I saw that the deadline for applying for the fellowship was coming up and I just quickly thought about what I could do. I’d been looking at other coding projects around that time and I’ve always been interested in creative coding. So, I thought it’d be nice to combine music which I’ve been doing for a little bit now and computer science or coding which I’ve been doing for a little bit longer to try and see what I could put together with the two of them. 

Angela: Yeah. So, when did you start getting into music and coding? 

Sean: I got into coding my sophomore year. I didn’t know that I wanted to be a computer science major. But as I’ve done it, the more I’ve done it, the more I have sought out doing coding projects outside of school. And I’ve been doing music since middle school where I was in band. But during the pandemic lockdown, I started looking at electronic music and people doing it on YouTube. So, I got inspired to try through that. 

Angela: So, do you use a specific program for that? 

Sean: I use Bitwig Studio which is a digital audio workstation. It’s very similar to Ableton Live, which is much more popular. 

Angela: All right. So, with that program, when did you start using that? Just, you started using that before the fellowship, right? 

Sean: Yes. I’ve been using it for a couple of years now. I’ve taken two music technology classes where I used that for personal projects, so I was already familiar with it which made it easier to come do an independent project with it.  

Angela: Okay, cool. So, when did you first hear about the DKC? And then like about the fellowship? 

Sean: I remember being introduced to the DKC in my first year. I think as a freshman I walked through here at some point. But I’ve become more familiar with it since I work in the HCC, so I’ve interacted with the DKC a lot more often, but I haven’t used it for any personal projects. 

Angela: Oh, where do you work? 

Sean: I work at the info desk. Yeah, I heard about the fellowship from watching people come downstairs and interact with the media wall. I remember seeing Josiah doing his almost photogrammetry project. I know it’s something else. It’s a neural something project, so I just one day asked – stopped him. I was like, hey, what are you doing? What is this? And then he told me about it. He told me that the fellowship was happening and what that entailed. 

Angela: And then I guess you sort of connected with Cartland and Shannon and proposed this idea, right?  

Sean: Exactly.  

Angela: All right. So, can you kind of walk us through your creative process in coming up with ideas? 

Sean: So, at the beginning of the project, it was just looking for ideas. I didn’t have a set goal in mind, so I just started messing around with music software in the way I have in the past trying to make kind of broadly ambient things. And I’m very interested in generative systems like Brian Eno and Steve Reich have used as grand historical examples. So, I was just playing around with those and then refamiliarizing myself with the creative coding side of it where in the computer science major, we’re very focused on building specific systems and working in very tangible ways where creative coding is less tangible and less goal oriented. So, I was just getting refamiliarized with that and getting familiarized with the JavaScript language which is used for it, which I haven’t used directly. So, it’s been relearning that. Part way through, I was looking back on old projects when I found something I started during lockdown, which was making rhythms out of the orbital periods of the planets where everything is a ratio based on the speed at which Mercury revolves around the sun. So if that’s my one, then what’s the ratio between that and the speed which Venus goes around the sun? Which is 2.5 and Earth is about 4 and then Mars is 7.75. So, I just went back to that project and was like, okay, it’d be easier to build off of something I’ve already started. I can go deeper into just that versus exploring broadly. And I remember at the time wanting to make a visual for it, but not feeling like I had the skills to do so. But now that I’ve gone further into coding, I felt more confident starting to do visuals for something I already had, so I’ve just been messing with visuals and thinking how I can represent this visually. 

Angela: That sounds really cool. So like, you kind of got inspiration from other electronic music artists and then as well as using your past project to kind of build off. 

Sean: Absolutely. 

Angela: Did you face any challenges when doing this? 

Sean: So far, my biggest one has been time management. This is a very heavy semester and it is hard making time for something that is not – that my graduation doesn’t hinge on. So, it has been very important to set aside specific hours of “Okay, a few hours a week at this time. This is my only concern. Just work on this project.” It’s something I enjoy anyway, so I should be able to sit down and do it. So, that has been my biggest thing. 

Angela: Gotcha. Do you have like some sort of organizational system to help with that or? 

Sean: A little bit. I’ve been, I let Shannon and Cartland know what hours I would be in the DKC, so they expect me to be here working for a specific number of hours. So, I always come in and like I’m here for these hours. This is all I’m doing for this time. I feel a bit more accountable that somebody else knows I’m supposed to be doing it at that time. And then, I’ve appreciated the check-ins I’ve been doing with them throughout the semester where I show up and I say here’s what I accomplished. Here’s what my next goals are. And it’s nice to be able to sit down and go, OK, this is what I’m looking to do between now and the next time I see them. So, having someone else know what I’m doing and not just working completely independently has been very helpful.  

Angela: So what is one thing that you’ve learned throughout this project? 

Sean: I’ve learned a lot more JavaScript. Very tangibly, I’ve learned about that. I’m learning just a variety of very technical things where I’m learning how to handle scope. In computer science this comes up a lot, something called scope creep where you have this idea that you want to accomplish and it’s very easy to simply keep making it bigger. At the beginning of the semester, I had really big ideas of what I wanted to do with this project and I very quickly realized I should shrink that because I’m not going to be able to put as much time into this as I want to. So, I’ve vastly simplified what I’ve been doing. And as I’ve been going on, I’m just continuously keeping that in line because I keep wanting to add extra features and make it much fancier and everything like that would require a lot more learning and a lot bigger time investment that I don’t have the time for at the moment. 

Angela: Where are you at his point in the project? And then where do you want to be toward the end of it? 

Sean: Right now, I feel like I’m 65 or 70% the way there. I have a presentable demo. Though, a lot of the rest of the project for me will just be refining it where I have all my rhythms from my music in place, but I don’t, I’m not happy with the way it sounds. And then, I have my visualizer working. It works with the rhythm, but I think it could look a lot nicer and it’s figuring out how to make it look nicer without introducing a lot of complexity. So for the rest of the semester, it’s just going to be refining and pushing it to be what I feel is nicer. 

Angela: Do you have plans to post this online somewhere or is this just for you?  

Sean: It will be posted online somewhere. I might make a version of this that will work on the wall. And I believe Cartland and Shannon want me to present it at the creativity symposium, I don’t remember what it’s called. 

Angela: Alright, well that’s cool. So once you graduate, do you plan to do similar things along these lines with the project and music and everything or is this just more like a fun hobby sort of thing? 

Sean: I think those can both be true. I’d like to continue working on maybe this project specifically or similar projects as hobbies in the future. Though, I don’t imagine this will directly tie into any like career aspects. I Imagine if I get better at coding as a software developer or something in that similar range, it will certainly have an impact on how well I can do something like this.  

Angela: So, what was the most rewarding thing about doing this project? 

Sean: I think the most rewarding thing has been the invitation to dive deep onto a singular project. It’s very easy to – I imagine many people, you know, it’s very easy to scatter myself across a bunch of projects. I’m interested in a lot of things, so it’s very easy to look at all the different things I’m interested in and not get very far in any of them. So, this project has been an invitation to sit down and focus on one thing and just get better at that singular skill. And that has been immensely rewarding because in the past I’ve been like, oh, I want to get better at creative coding. I want to make these kinds of projects. And I don’t sit down and give it the time to actually get further into it. So, this has this has been a very good time for doing that. 

Angela: And then another thing I wanted to ask, so you’ve been learning a lot on your own. You know, learning based off like previous projects and things like that. Have you had anyone to lean on to get help during this project or do you use like Internet resources? How do you learn? 

Sean: I’ve been using a lot of Internet resources. There are a lot of great YouTube channels that cover creative coding. The Coding Train is a phenomenal resource for that. And there is the same thing with music production. With Bitwig there’s a lot of great forums and YouTube channels for it. I talked to Dr. Whalen and JD about it a little bit because they both have interests in data visualization as an art form. So, it’s been nice to bounce some ideas off of them and they’ve come back with things that would again make my project much more complicated but were very inspiring to talk to them about. 

Angela: And with like the more visualization of this project, you wanted to post it on the HCC screen. So how does that go about? 

Sean: That comes with the coding. At the moment, my project is essentially just a bunch of circles moving around in a very specific pattern. But because it’s written in code, I can define the dimensions of the canvas by width and by height. So, all I have to do is change it to match the dimensions of the media wall downstairs and the entire project should just shift its aspect ratio and work perfectly fine with it. 

Angela: And then finally, would you recommend that students do a fellowship? 

Sean: Absolutely. I think students in general underutilize the DKC. I see that a lot down at the info desk where people come in and oh, I didn’t realize we could check this out. And like you have no idea what you can check out of the DKC. So, I know a lot of folks who are interested in doing digital projects or would probably enjoy making their own website where they have the opportunity with domain of one’s own. And I’m sure there are other students who are working on individual projects where they could work on the projects they’re already planning on doing, but with extra structure or support or somewhere to present it like the wall or like the creative symposium. 

Angela: And you mentioned like maybe posting this on the Internet, so would that be like on a website or are you thinking more like a video like YouTube video? 

Sean: At the moment, I’m just thinking a YouTube video. If I end up with more projects like this, I’ll probably end up making my own website and where it would be hosted directly. 

(Outro music comes in and fades out) 

Angela: This has been the Digital Creator Podcast. You can find a transcript of our conversation and more about this project below. This podcast was produced by me, Angela Payne, with help from the resources at the Digital Knowledge Center. Are you interested in becoming a digital creator? Do you have a cool project you want to share? Go to dkc.umw.edu for information on how to get started. Thank you for listening! 

(Outro music fades back in) 

For information on how to apply for a fellowship visit the DKC site.

Posted in

Reply

Your email address will not be published. Required fields are marked *