Frequency Domain public builds are up! Including Oculus Rift builds!

Builds are available for PC and Mac.
The recommended experience is PC + gamepad (barrel rolling doesn’t work right on Mac right now)

I got some hands on time for the time this past Tuesday with a, Oculus Rift devkit. I was able to hack up a build that day, it’s very rough and needs a ton of work, but it is basically functional.

Normal builds links:
http://steamcommunity.com/sharedfiles/filedetails/updates/142393387/1369611489

Oculus Rift builds links:
http://steamcommunity.com/sharedfiles/filedetails/updates/142393387/1369807813
I highly recommend visiting the Steam Greenlight Concepts page for the latest info on Frequency Domain:
http://steamcommunity.com/sharedfiles/filedetails/?id=142393387

 

Frequency Domain v0.2 (ft. Daft Punk – Touch – Random Access Memories )

Added the last major feature before the Indiecade submission build:
Three semi ring like structures that each reflect the overall activity on different parts of the frequency spectrum.

More info about the game on the Steam Greenlight Concepts page: steamcommunity.com/sharedfiles/filedetails/?id=142393387

(Feedback would be really appreciated!)

 

Youtube version:

 

Vimeo version (looks slightly better):

 

 

Frequency Domain v0.1 (ft.Daft Punk – Around The World / Harder Better Faster Stronger – Alive 2007)

Introducing the new feature I’ve been working on since last week: flying creatures! (I don’t know what to call them yet -__-)

If you observe carefully, you’ll see that these creatures after a player performs a significant jump. They’re movements reflect what the player did during the hangtime of the jump and the colors correspond to the colors of the music that were playing during that jump.
You could say it’s a time travelling tool that shows you what you did and in what musical context.

Music: Daft Punk – Around The World / Harder Better Faster Stronger – Alive 2007 (I edited the beginning of the song a little to get to the interesting part faster)

Frequency Domain Playable Prototype 4 (ft. Daft Punk – Get Lucky [Radio Edit])

I was originally going to use the 1 min teaser they had up, but then the 4 min Radio Edit got out!
On the game side, I’m working on a big new feature, but it wasn’t ready to show yet, so I’m sticking to a slightly older feature set build for this video.
Main things gameplay things here are the barrel roll and “warping”, plus some minor camera tweaks.

Sorry for the slowdown at the beginning, it stable for the rest of the video though.

YouTube version:

 

Vimeo version:

 

 

 

Frequency Domain Playable Prototype 3 (ft. KORG volca series trailer music)

Saw/heard the KORG volca video the other day, I just had too see what it looked like in game :D .

Minor camera tweaks, mostly fixing core back end stuff, wanted to get that out of the way before I start working on the new features.

(Apologies for the occasional audio/video distortions, the recording process had some hiccups)

Music from the KORG volca series trailer video: http://www.youtube.com/watch?v=_Gau7UIzYY0&feature=youtu.be

 

Youtube version:

 

Vimeo version:

Save tons of ms by NOT scaling your mesh!

I was looking at my profiler yesterday looking for something that I had seen several times before, but never really bothered to act on: the “MeshRenderer.ComputeScaledMesh” function.

This screenshot of the profiler show it taking up to 3.89 ms on my PC (decent 4 year old gaming rig).

FD_Profiler_WithScaling

It didn’t really make sense, since the scaling was minimal ( 3 on the x axis, 2 and on the axis) and only applied to one object, the main field mesh. In fact, that’s the only thing I was rendering in the game, why use up almost 4 ms !?!?!

A little googling later, I found out I wasn’t the only one experiencing this issue and the “fix” was to simply not to scale >__< . Lucky for me, I didn’t change the scaling during gameplay and I could easily “pre-scale” the mesh when I first generate it.

And here are the results:

FD_Profiler_NoScaling

 

As you can see in the profiler, the ”MeshRenderer.ComputeScaledMesh” completely vanished. The game now runs between 4 and 5 ms faster than before overall! And that’s on the PC, imagine the relative speedup on a laptop (one report puts it as going from 30 to 70 fps!!!) !

Frequency Domain Playable Prototype 1 (ft. Aliceffekt)

FD_TES2_gif1

This new update is the biggest leap so far, with a ton of stuff I’ve finally implemented:

- Now doing proper inter-vertices interpolation for height values (i.e. smooth movements)
- Implemented a pseudo-log scale that allows me to encompass the entire spectrum (if desired). Data resolution tweakable on a per decade basis, on the fly
- Implemented basic velocity, friction and gravity (still needs a lot of polish)
- Implemented basic camera movements based on player (still needs a lot of work)

Visually, I’ve added a hint of bloom and motion blur (thanks to Unity Pro!), otherwise it’s pretty much the same “graphics setup” at the previous tech video.

Here is the gameplay video of the gif above, featuring a track by Aliceffekt!

 

And since YouTube isn’t playing nice, here’s the Vimeo version

A Technical Breakdown of Frequency Domain

Here are some of the core technical elements that make Frequency Domain possible.

Getting the frequency data

The core of Frequency Domain is the Fast Fourier Transform (FFT) data being used to generate the terrain. The FFT data is stored in an array, where each element represents a “bin”/frequency and the value of the element (ranging from 0 to 1) represents the amplitude of said frequency.

Unity makes it super easy to get FFT data from audio, either from a single audio source or from the main audio listener (what the player hears, all audio). Frequency Domain uses the single audio source variant of the function :  AudioSource.GetSpectrumData 

Generating the mesh

The terrain is generated at runtime, in code. The Unity website has a great guide on how to create a mesh from scratch here. It basically comes down to 2 main steps: generate a list of vertices, then connect said vertices into triangles.

Here is the bit of code that handles it:

List<int> trianglesList = new List<int>();
        List<Vector3> verticesList = new List<Vector3>();

        Vector2[] uvArray;
        List<Vector2> uvList = new List<Vector2>();

        // initial line
        for(int j = 0; j < verticesFrequencyDepthCount; j++)
    	{
    		verticesList.Add( new Vector3(0,0,j) );
    		uvList.Add( new Vector2(0,0) );
    	}

    	// populate the rest of the vertices, triangles
    	// use verticesFrequencyDepthCount to shift between frewuency collumns
        for(int i = 1; i < verticesTimeDepthCount  ; i++)
        {
        	for(int j = 0; j < verticesFrequencyDepthCount; j += 2)
        	{
        		// bottom left triangle
        		verticesList.Add( new Vector3(i,0, j) );
        		int currentListIndex = verticesList.Count -1;

	        	trianglesList.Add(currentListIndex);
	        	trianglesList.Add(currentListIndex - verticesFrequencyDepthCount);
	        	trianglesList.Add(currentListIndex - verticesFrequencyDepthCount + 1);

	        	// fill triangles in between this and previous triangle below
	        	if( j > 0) // is not at the edge
	        	{
	        		// bottom left triangle
	        		trianglesList.Add(currentListIndex -1);
		        	trianglesList.Add(currentListIndex - verticesFrequencyDepthCount -1);
		        	trianglesList.Add(currentListIndex - verticesFrequencyDepthCount);

		        	// top right triangle
		        	trianglesList.Add(currentListIndex);
		        	trianglesList.Add(currentListIndex - 1);
		        	trianglesList.Add(currentListIndex - verticesFrequencyDepthCount);
	        	}	

	        	// top right triangle
	        	verticesList.Add( new Vector3(i,0, j + 1) );
	        	currentListIndex++;

	        	trianglesList.Add(currentListIndex);
	        	trianglesList.Add(currentListIndex - 1);
	        	trianglesList.Add(currentListIndex - verticesFrequencyDepthCount);

	        	uvList.Add( new Vector2(0,0) );
	        	uvList.Add( new Vector2(0,0) );
        	}
        }

        verticesArray = verticesList.ToArray();
		trianglesArray = trianglesList.ToArray();
		uvArray = uvList.ToArray();

		Debug.Log(trianglesList.Count);

		mesh.Clear();
		mesh.MarkDynamic();
		mesh.vertices = verticesArray;
		mesh.uv = uvArray;
		mesh.triangles = trianglesArray;
		mesh.RecalculateNormals();

Faking physics

My initial plan was to simply use Unity’s built-in physics engine, but it ended up being too slow to use on the entire mesh at once. I was tempted to use smaller sub-meshes instead, but it didn’t feel like a very elegant solution.  I soon realized that I only really needed the height information of the player, and so using the vertices list as a heightmap would suffice.

Here’s the function that handles it:

public float getHeightFromPosition(float xPos, float zPos)
	{
		float height = 0;

		float xScale = transform.localScale.x;
		float zScale = transform.localScale.z;

		//normalize postion to a unit scale
		xPos = xPos / xScale;
		zPos = zPos / zScale;

		float xFloor = Mathf.Floor(xPos);
		float xCeil = Mathf.Ceil(xPos);
		float zFloor = Mathf.Floor(zPos);
		float zCeil = Mathf.Ceil(zPos);

		//make sure position is within mesh
		if( xFloor <0 || xCeil >= (float)verticesTimeDepthCount  )
			return 0;  //return height 0 is out of bounds
		if( zFloor <0 || zCeil >= (float)verticesFrequencyDepthCount  )
			return 0;

		// get height of 4 corners arround position
		float TL = getHeightFromHeightMap( (int)xFloor, (int)zFloor );
		float TR = getHeightFromHeightMap( (int)xFloor, (int)zCeil );
		float BL = getHeightFromHeightMap( (int)xCeil, (int)zFloor );
		float BR = getHeightFromHeightMap( (int)xCeil, (int)zCeil );

		float xLeftLerp = Mathf.Lerp(BL, TL, (xCeil - xPos ) );
		float xRightLerp = Mathf.Lerp(BR, TR, (xCeil - xPos) ) ;

		height = Mathf.Lerp( xLeftLerp, xRightLerp, zPos - zFloor );

		return height;
	}

 

Converting to a pseudo-log scale

The FFT data coming from Unity is linear, i.e. all the frequencies are equally spaced out from one another. Unfortunately, audio/music “notes” as we hear them are not linearly space out, they’re more on on a log scale. In order to represent the entire audio spectrum at maximum FFT resolution (8192 data points), I had to implement some sort of logarithmic scaling to the linear data I got from Unity. The following piece of code allows me to convert those 8192 data points into 100.

// get raw FFT data 
		audioSourceArray[0].GetSpectrumData(sampleArrayFreqBH, 0, FFTWindow.BlackmanHarris);//Rectangular);

		// cleanup pseudolog array first
		for(int i = 0; i < pseudoLogArray.Length; i++)
			pseudoLogArray[i] = 0;
		
		// doing the pseudo log scale
		int decadeIndex = 0;
		int fftSampleCounter = 0;
		for(int i = 0; i < pseudoLogArray.Length; i++)
		{
			if( i != 0 && i%10 == 0)
				decadeIndex++;

			for(int j = 0; j < samplesPerDecadeArray[decadeIndex]; j++ )
			{
				pseudoLogArray[i] += sampleArrayFreqBH[fftSampleCounter];
				fftSampleCounter++;
			}
		}