rhoadley.net   music   research   software   blogs

aru    seminars    m&t    critski    focm1a    cmc    circuit bending    mic2b    sensor technology    comp 3    sonic art    major project
youtube    vimeo    facebook

Resources:    Bioacoustics    Jitter    MaxMSP    OSC    Physical    PD       CBHH    sTech    SuperCollider    C/Xcode

Jitter Resources:     Home     Blog     Forum     Examples     Projects     Tasks     Tutorials

Jitter Tasks

The Image to Sound Task

Through a series of tasks, develop a simple demonstration of image to sound conversion. Part 3: Video Analysis

Tutorial Patch

Task 3 Name: Image to Sound 3 Set: w2 Due: w3 Weighting: 0% Courses: cmc2b
Prev Task: Image to Sound 2 Next Task: Image to Sound 4
Task Summary All cmc2b tasks VLE

Also to cover

  • Recording movies from jitter objects (jit.qt.record)
  • Routing audio from movies to Max/MSP (spigot~)



The Task

Video Analysis

  • This part of the task involves using each part of the video you've extracted to give you information which you can use in sound synthesis.

  • An example of what you can do is to create a patch that calculates a number between 0.0 and 1.0 (or 0 and 255) based on an incoming video stream. For instance, the number might be 0.0 (zero) if the video stream is not changing, and 1.0 if it is changing a lot. Or zero if the video stream is black and 255 if it is white. You could determine a sound from a number obtained by adding the number of pixels that have changed in the current frame of the video. Clearly, there are many options here, and the detail is up to you.

  • Recommended objects: jit.3m, jit.histogram, cv.jit objects, jit.spill, and others...

  • cv.jit objects are available here. The CV stands for Computer Vision and they include a lot of nice functionality especially related to motion tracking. Keep these in your own directory and choose this within File Preferences in Max/MSP as when dealing with any external objects you've downloaded yourself. There's a overview to get you starteed.

  • You might want to test your analysis patch on a standard video stream (eg. quicktime movie), and plot the results using multislider.

  • Patch samples
    Please ensure that, in as far as you are able, your patches work immediately they are opened. If your patch or patches use audio or video files, include short samples that have been 'preloaded' or reference to a stored one. Also include methods for loading user samples, too. Include functioning presets.

  • Media files
    You must submit media files, such as video, audio or image files, but please ensure that video files are compressed to a reasonable degree. You should never submit dv files, but compress these to mp4. You should submit no file that is greater in size than 25MB/minute.

  • Added value
    By completing the details of the task you will achieve at least a pass mark. By imaginatively and creatively considering how you might implement the task originally you can add value to your submission, and this added value may increase your mark significantly. Even when making videos of short demonstration tasks try to consider musical and performance criteria.

  • Don't just investigate the help file for a particular object - investigate the help files of the linked objects as well. Using these, developing your ideas into something individual and unique will give you many more points and may well help you decide what you want to do for your final project(s).

  • Find the music
    One particular way of adding value is to make sure that your submission is satisfying musically as well as working technically. Many people are able to make the technologies work, but fewer are able to make them work in a way that enhances the musicality of the result. No matter how technically brilliant, a submission with poor musical results is unlikely to do so well.

Finally

  • Compress (zip) your patches, demos, etc. into one file called your_student_number_"jitter3" (e.g. 0504335_jitter3.zip), include a readme with your name and student number and, if necessary, how to use or just open the patch.

  • Submit a copy of the files to the i-Centre on 2pm on Tuesday 8th May 2018

You might also be interested in:

  • Jitter tutorials

The Projects

The projects and tasks are designed to help you through the various courses and materials that you'll have to deal with, and also to provide an active and practical element to what could otherwise become a rather dry and technical exercise. Tasks are small exercises - you may be asked to complete one or two per week. Projects are larger and carry a higher percentage of the mark. We will undertake two, three, four or more projects and tasks. The final project is usually an individual choice project, and will be worth significantly more than the others in terms of percentages in your portfolio. We will usually try to set aside a time to perform the projects in a public setting.