Real-Time Social Spaces



Week 14- Final Presentation


Questions after the presentation:

  • Who is the audience for this project? How can strangers interact in this space? Can you create rooms, so that people with the link can only join?
  • Can this be a space where you can eat in silence? Is this about having a conversation or eating with other people so that you don’t feel alone?

I mostly thought about this space from my perspective, not really thinking about others. I think this could be a private or a public space. What if you can create rooms and only people with that link can join your session/room? This would give a more private space for families and friends. I mostly do this during playing the game Gartic.io. You can either play with strangers or create your game room. 
The space has audio from people’s  microphones. But I can get how that can be too distracting with echoing sounds etc. There can be a mute button so that people has the power to either join with their audio or not. 


Overall, this project is a start. It took a quite some time to understand the basics of a server. But I can proudly say that I learned a lot from this process. The code helped me learn more about class syntax, starting a server etc. I need to think more about the interaction side to dig deep into the idea of “eating together”. 

  
I created a sphere for the background and assigned a double-sided image (which I created using skybox AI) to work like a skybox. I also added some fog to create a more dreamy enviroment. The background is simple and clean. 


One of my problems if how the videos freeze once 5 people are in the room. And once it’s 4 people, the webcam’s are delayed. The image above kinda shows how in once image I’m moving my hand and the other’s are just frozen. This might be a problem because it’s my entering the room with my camera but I believe that hosting multiple people in a room with webcam can cause problems. 

As for the next step, I want to add planes to the table and people can add their own image (their food pic). Spencer had a similar thing where people can upload audio into the room. Maybe his code can give me a sence on how to upload files into a space. 
 

 


Week 12/13- User Testing


Project Description

The “Virtual Dinner Party” project aims to recreate the experience of dining with loved ones who are physically far away. It will capture the essence of socializing around the dinner table and provide a unique way to connect with long-distance family members and friends through shared experiences of food and conversation. The project involves creating an online-social environment, where users can join with video and their videos will fir into virtual chairs around the table.


Important shortcuts:

To exit the server: ^C
To restart the server: node server.js


Understanding how a server/client works definitely took me quite some time. The class recording was really helpful in terms of going back and watching Aidan draw a diagram how a socket works in this context.

Week 13 Feedback Notes: (From Yu-ying, Cindy, Austin)

Leave the background as simple as possible. Even just a color is enough. This way, people can focus on the dinner table.
My initial idea was to assign seats to people once they join the server.  But they suggessed me to add another functionality where people can change seets etc. 
Maybe a small functionality where people can cheer by dragging a cup can be a nice touch. 




So, I didn’t assign anyone a seat. Clients can join the “room” and choose their own place by the arrow or wasd keys.  
I mostly don’t use a class syntax but for this example I had a separate js file to separate the three.js environment and the WebRTC stuff. Loading a 3D model in the had a different syntax under class structure. It took me a loooong time to make it work. Then, this person saved my life! 



What is WebRTC?

An open framework for the web that enables Real-Time Communications (RTC) capabilities in the browser.WebRTC is a protocol that allows you to create real-time communication between peers using JavaScript. Thus, to import a webcam texture, I had to use WebRTC. 


Question:
I’m using THREE.DoubleSide() but the image is only shown in one side?


Github Repo (yay! I finally learned how to commit code to github!)

Final Presentation Slides





Week 11- Final Project Prompts

IDEAS

Since I’ve moved to New York,  I have to think twice before I call my parents. I try to catch them during dinner time so that I can talk to them both. This turned into a routine where I call them after my class during 12 PM and it’s 7 -8 PM in Turkey. They place the phone just across where they are sitting. This way, I get to see them and the dishes, feeling like I’m also there with them. My family values eating together and the conversations around food. Therefore, even being in this virtual environment with them makes me feel better. 

For the final project, I want to re-create this social environment that builds upon my home. I want to capture the essence of being around the diner table by placing users around it. Ideally, I want them to join with video and upload the image of what they are eating. The images would be placed on the table, and their video capture would fit on the chairs in this virtual space. 

“Virtual Dinner Party” is a way of coming together with family members or long distance friends around food. The virtual space will have a spacial audio of a house. I wish I could make a auto-extending table where when the number exceeds the number of chairs, the table would extend and make space for new commers.







Week 9/10 - Node.js

I’m still not entirely sure what “node.js”  is. The description: “Node.js is a cross-platform, open-



What is strange to me is how you can’t download a script from github. I have to manually copy and paste it to my computer and make a folder. Otherwise, if I say download, I have the html page. 

After creating the documents, I was able to run the server!



Excited to see what is next!!!




Week 8- Interactions and Controls

EXPLORING SPACIALITY

I enjoyed John Palmer’s talk on Spatial Softwares. It helped me distinguish between different softwares and their approach on spatiality and virtual bodies. Also, I definitely experience playing more games. While trying to design products, I would benefit learning from different environments and the interactions. 

He also made a clear distinction between dimentionally/spatiality and being “better”. Apparently, if you have an avator as a cursor in Figma and moving your avator with controls doesn’t make any sense. Every little detail is designed thinking about the user experience. The virtual body should be aligned with the space. As a user, you have to feel included within the space and it should follow the purpose.  

DATA PORTABILITY

I got the part where the article talked about standardizing file formats (gltf, USD). But I’m still not sure how might moving a content from one platform to another would benefit a user. Both the articles emphasized exporting the avatar. If I did that, where else would I import it? Each platform has different design aesthetics and visual feel to it. The avatar I created for one might not be in relation to another one. Basically, I’m not caught up with this whole interoperability term. 




Week 5-6-7- Agency and Controls

MY CODE NOW EXISTS ON GITHUB

Link to the github repo: I WISH

Wohoo! I don’t have to copy and paste my code to notes app from now on! I created my own repository and saved the changes in github. Definitely a game changer. 

Also, the overall syntax of three.js was a bit confusing to me. I was having trouble incorporating the example code from the RTSS repository and merging into my code. After countless trials, I think I’m getting there. 

My office hour with Aidan was really helpful in terms of solving my problems from Week 4. 
1. gltf.position.set does not work! The correct syntax is gltf.scene.position.set();
2. The audio was working, but apparently safari is not playing the audio. Using chrome solved the issue. Now, I added the html preview screen to VS Code which allows me to use see my sketch without going live.




Questions from this week: 01. Is there another way for scaling objects rather than using object.scale.set()? I feel like it’s not doing what I want it to do, the position of objects change once I set a different number.

02. How can I cut a surface from a plane or a boxGeometry? I wan’t to put the window model into it. 



03. Since we added the video to html first, the video is being previewed on the top. How can I delete it? 


04. Is there an easier way of locating the directional/point lights? I want the light to come outside the window blinds. 

Aidan gave a couple options for the water surface. I couldn’t make the water shader and water texture to work. So, I used a videoTexture for the material for the ground. I want it to feel more real and more layered. For next week, I want to experiment using the raycasters and click actions. Once I have more objects etc. I will use the class syntax to have a more organized code. Even now, I have troubles finding something.

During our class discussion, Vesper gave me good resources to solve my problems:
- Using clipping mask: Ex
-Paul Chan’s light art alligns with my vision.
-Using Lensflare for lighting.





Week 4- 3D Models and Spacial Sound

MEMORY PALACE / DREAM SPACE

I mostly enjoyed exper




I found Porter’s environment similar to my overall visual aesthetic, with shadows of windows. First, I thought that the shadows are created with physical windows, but apparently the room is created with imported textures. I loved how he talked about “memory palace”, it made me think about Sherlock’s mind palace. 



Overall, the rooms made more sense when I listened to the audio. Otherwise, it’s harder to experience it. I might think about recording my own voice and talking about the room when I can create a full experience ︎

GLTF LOADER

  

I spent an enormous time just on importing a 3D asset. I wanted to create my own “memory palace” with an olive-tree and a plactic white chair. For me those two represents a sunny Mediterranean day; you hear the birds chirping and a little breeze.
The objects that I dowloaded were  - .gltf -. Even though I used the code in the examples section, it didn’t show up. First, I thought maybe I had the code wrong or smtg. So, I tried the toad example and I was able to see the animal. Apparantly, .glb worked but not .gltf. I had to find new/free glb objects. Below, you can find the links:


olive
chair

I have several questions: What might have been my mistake while trying to import a gltf file? It was really annoying to see how easy was it to import a glb file. Also, once I have the object, I wanted to position.set() it, but I was not able to change it’s position.

--


Then, I wanted to attach a background audio to the camera or have a background sound. I used this code:

// create a global audio sourcesound = new THREE.Audio( listener );
// load a sound and set it as the Audio object's
bufferaudioLoader = new THREE.AudioLoader();
audioLoader.load( 'relaxing-garden.mp3',
function( buffer ) {
sound.setBuffer( buffer );
sound.setLoop( true );
sound.setVolume( 10 );
sound.play();});


This is my audio file: sound

The code works, but I can’t hear anything.  

References:

GLTFLoader
AudioLoader
Audio
How to add background music?
Loader example
Importing 3d models

*https://threejs.org/examples/#webgl_geometry_terrain




Week 3-Transform Hierarcy and Add-ons

VR THEATER

As group #1 (me, Lan, and Elodie), we found the topic of VR theater very interesting. After reading this blog post from SIGGRAPH (conference for computer graphics and interactive techniques worldwide), we dig more into how the concept of VR theater is changing the traditional theater performances. 

VR Theater is an immersive storytelling showcase celebrating the evolution of VR with a selection of short-form narratives in an in-person theater setting. It showcases of exemplary virtual reality storytelling. The main program will be experienced in a high-end, seated, panoramic, multi-viewer immersive space, running approximately one hour from start to finish. 

It influences the theater experience in many ways. Such as, VR provides a 360° experience that lets audience feel like they're part of the scene, with the ability to view everything from every angle (Virtual Presence), VR allows the audience to emotionally engage with the character by using the point of view (POV) narrative (Character Embodiment), and last but not least, Modern VR headsets with motion trackers enhance immersion by letting users move freely in the virtual landscape (Freedom of Movement).


When we first heard the term “VR theater”, we thought that it was a new way of performance and believed that it emerged during the covid era. After doing some research, we found out that we were wrong! The First Immersive Virtual Reality Movie. "ANGELS" was conceived at MIT & developed at the Hitlab in Seattle. It was completed in December 1991 and recorded on video in January 1992.And, one of the leaders of VR integration with immersive theatre is the National Theater of London. In 2016, the NT launched its Immersive Storytelling Studio to explore how virtual and augmented reality could feature in productions to enhance the audience experience. 

How did covid affected theater performances?
Immersive theater experienced a dramatic increase in engagement in the wake of the pandemic. With real theaters closed, actors are turning to virtual reality. The addition of virtual & augmented reality to immersive theater provided an avenue for consumers to enjoy these experiences from the comfort of their own homes.

In Zoom (or other video-based) performances, everything is about actors and their faces. In VR, theater is the opposite: no real faces, but lots of physical movements. With hand controllers and a headset that tracks head movements, it can become like dance.

The future of theater is headed to a place where I can’t fully envision right now. For me, theater is a live performance where you’re surrounded by the actors and actress’s and feel the adrenalin. You are captivated by a performance withi a short time period. Each play is different from one another because even though the script is the same, how people and the audience is different and this affects the overall mood of the play. So, the term VR Theater doesn’t make fully sense for me. I feel that “theater” will change its meaning in the future. 

For the presentation, click here!



ADD-ONS

I was relieved when we talked about orbit controls in class; because, it was really hard to locate the camera by eye-balling its position. I watched the recording and repeated step-by-step. Now, I have both the gridHelper() and the orbitControls() add-ons. I can easily pan and see my sketch from each angle.

   

The black background is killing the overall vibe of the environment. I’ve been meaning to learn how to change it. Luckily, we covered that part in class this week as well. I watched this video, and figured out to add a skybox. To be honest, it took me a while to understand the image specifications. Now, I have an image for each side, but how can I make a cut-out version image? Rather than having 6 different image, how can I make the image below?



Well, I added a skybox and the scene now looks like this! It’s not the best image to use, but at least I managed!







Week 2 - Light and Shadow

SPACES AFFECT HOW PEOPLE INTERACT WITH ONE ANOTHER

I rely on the subway for my daily commute, using it at least twice daily. This means the train has become a familiar and frequently-used physical space for me. It features two rows of seating, as well as grab bars for those standing. Advertisements and posters decorate the walls, while a display panel shows the route and announces the name of each station. This panel is especially useful for those with disabilities or for people wearing headphones, allowing them to stay informed of their location and direction of travel. 

I trust the subway because it consistently provides accurate information. The signage is clear and easy to understand. However, when I take the A train, I find the lack of signage uncomfortable. To determine my location, I must either listen carefully to the conductor's announcements or have good eyesight to read the station names displayed when we stop.

Eye contact is the primary mode of communication on the subway. If I don't understand an announcement, I will engage in conversation with others. Additionally, I may ask someone to step away from the door if I need to disembark. Otherwise, passengers tend to keep to themselves, engaging in activities such as reading books, browsing their phone, watching TV shows, listening to music, or reading newspapers.

FINALLY I’M USING VISUAL STUDIO

It took me a while to understand how visual studio works, and I had a hard time opening the local server. For this exercise, I chose this photo from the Library of Congress archive: (link) I love the shadow of the circular window. Since our theme was light and shadows, I wanted to recreate the same shape and shadow on a plane. 

Early iterations:

I spent most of my time by adjusting the position of the directional light. 

  

Then, I added textures to the floor and wall. It completely changed how the room looks and feels!

 

Questions:

    1. Can I rotate a directional light?
    2. Can we change the color of the background (black)?
    3. How can I add a library into three.js (gsap library of animation)?
    4. How can I move my camera back and forth with subtle motion?


***this guy makes this code work, but I can’t. Why is that? (link)

let z;
const zFinal = 14;
z = camera.position.z;
    function animate () {
z+= 0.1;
if(z < zFinal)
camera.position.z = z;
renderer.render(scene,camera);
}
renderer.setAnimationLoop(animate);





Week 1 - Basic Setup


BEYOND BEING THERE

The article feels really out-dated since the notion of “being there” is already accomplished by video and audio chatting platforms that exist today. It was quite funny to read how the technologies that we are used to in today’s reality and maybe even use every day now, are seen as something imagined in this article. It was also quite surprising to me when I heard people’s reactions to Steve Jobs’s first iPhone introduction. Considering that there was no touch screen feature until a decade ago, we actually need to think about the existing technologies and appreciate them a little more.

The article discusses how people would choose the natural, the real thing over the imitation, since face-to-face communication provides a richness of interaction that can’t be matched by any other method. I highly doubt that after experiencing the covid-era. The article would definitely feel different if I were to read this article before covid. I feel like the way people define communication has changed since we used Zoom for work, for class and even for group meetings. People’s habits have changed for sure. They might find using telephony products more practical. The “proximate reality” that face-to-face interactions discussed in the article can be achieved by other ways of communication too. The notion of “reality” is somehow different now and the computationally-mediated communication is a new medium.


KINETIC SCULPTURE

p5 sketch link : Kinetic Sculpture

Alexander Calder's colorful sculptures have always fascinated me. For this assignment, I was inspired by his works. I sketches out a designn with primitive shapes: sphere, cone, cubes, and a cuboid.

I learned that we can use the same material with different elements. We don’t have to create a new material for each new object. All I have to do is using the same material variable when I’m describing the mesh properties.

new THREE.Mesh(geo, mat);

Just like rotation, it’s pretty easy to change the position of an elements. I just have to calculate the right x,y,z values for the object, or just eye-ball them.

mesh2.position.x = -3; 
mesh2.position.y = 1.7; 
mesh2.position.z = 0;

   

Here is the final design! I tried to keep the same visual language by using similar colors. 



Questions:

  • How can we change the color of the background?
    The spheres I made doesn’t look right? What’s wrong with them? (they don’t look like a sphere when they are turning around)


links that saved my life:

Cone Geometry
RRGGBB values for colors
Position of objects
Sphere Geometry
Adding many elements to the scene

















 
Lipsum™
Integer turpis massa, varius ultricies est sed, aliquet feugiat enim. Nulla facilisi. Ut in justo in ex faucibus ultricies. Proin id erat quis lorem vehicula commodo. Phasellus pharetra libero pretium ultricies aliquam. Vivamus eleifend, ligula ut venenatis aliquet, massa elit efficitur lorem, fermentum molestie elit ligula in ex. Duis mauris tellus, pharetra et orci a, aliquam imperdiet augue. Pellentesque elit neque, tincidunt sit amet imperdiet eget, porta ut metus. Nulla id leo tortor. Donec elementum ac sapien eget feugiat. Donec lorem dui, vestibulum quis nisl et, feugiat mattis massa.


Phasellus Rez

Praese
Suspendía

Lipsum Deus

  1. Praese Vita
  2. Nulla Vehicu AI
  3. Ukam Cors
  4. Dictum Magna
  5. Cursus Ex
  6. Suspendis Est
  7. Facili Sap
  8. Et Magni Disc
  9. Phasellus Deus
  10. Deus
  11. Rexus
  12. Maximus
  13. Lorem Ipsum
  14. Dolor 123




©MMXXI
Integer turpis massa varius