The Colour of Music/the music of colour

色彩理論に基づく音楽信号の可視化、および色入力に基づく生成音楽システムの構築を探究する研究。

Explores the visualisation of musical signals based on colour theory and generative music systems based on colour input.

Research papers

  1. Haimes, P. (2025). Chord Colourizer: A Near Real-Time System for Visualizing Musical Key. In Proceedings of the 23rd International Conference of Asia Digital Art and Design Association, Tokyo, Japan. https://www.jstage.jst.go.jp/article/adadapic/2025/0/2025_CA0501/_pdf

    (This research received a “Best paper” award at ADADA 2025 conference!)

  2. Haimes, P. (2025). Colour Chordifier: Interactive Colour-Driven Music Generation. In Proceedings of the 23rd International Conference of the Asia Digital Art and Design Association, Tokyo, Japan. https://www.jstage.jst.go.jp/article/adadapic/2025/0/2025_CP2501/_pdf

 

Aesthetics In HCI

HCI分野で、美学をより深く考える重要性を提唱する研究。

This research advocates for a deeper philosophical engagement with aesthetics in HCI (Human-Computer Interaction) discourse. Interactive display by Moment Factory.

Research paper

  1. Haimes, P. (2021). Beyond beauty: Towards a deeper understanding of aesthetics in HCI. In Extended abstracts of the 2021 CHI conference on human factors in computing systems (pp. 1-8). ACM. https://doi.org/10.1145/3411763.3450381

 

FUJI-Chan

富士山の噴火データおよび気象情報を提示するためのアンビエント・メディア・システム。

An ambient media system for the presentation of volcanic eruption data and weather information for Mount Fuji.

Research papers

  1. Medley, S., Zaman, B., & Haimes, P. (2020). The role of cuteness aesthetics in interaction. In R. Rousi, J. Leikas, & P. Saariluoma (Eds.), Emotions in technology design: From experience to ethics (pp. 125-138). Springer. https://doi.org/10.1007/978-3-030-53483-7_8

  2. Haimes, P., Baba, T., Suda, H., & Kushiyama, K. (2017). Fuji-chan: A unique IoT ambient display for monitoring Mount Fuji’s conditions. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys ’17) (pp. 246–249). Association for Computing Machinery. https://doi.org/10.1145/3083187.3083223

Japanese DESIGN aesthetics

日本の美学が、個人の創作活動や生活環境の共同的な形成にどのような影響を与えるかを探る研究。

This work explores how Japanese aesthetic principles inform both individual creative expression and the collaborative construction and maintenance of lived environments.

Research papers

  1. Haimes, P. (2022). Machizukuri (neighborhood making): Collective world-making in traditional Japanese neighborhoods. Contemporary Aesthetics, 20. https://contempaesthetics.org/2022/11/10/machizukuri-neighborhood-making-collective-world-making-in-traditional-japanese-neighborhoods/

  2. Haimes, P. (2020). On Japanese minimalism. Contemporary Aesthetics, 18. https://contempaesthetics.org/2020/09/24/on-japanese-minimalism

  3. Haimes, P. (2015). Zen and the art of website maintenance. Interactions, 23(1), 20–21. Association for Computing Machinery. https://doi.org/10.1145/2847596

Virtue ethics in Interaction Design

適応型AIシステムなどのインタラクティブ体験において、美徳（virtue）がどのように基盤となり得るかを検討する研究。

Examines how virtues can underpin interactive experiences, such as in adaptive AI systems.

Research paper

  1. Haimes, P., & Uhde, A. (2025). Virtues in Gamified Systems: Exploring Virtue Ethics in Cross-Cultural Adaptive Gamification. In Proceedings of the 23rd International Conference of Asia Digital Art and Design Association, Tokyo, Japan. https://www.jstage.jst.go.jp/article/adadapic/2025/0/2025_CP1401/_pdf

NAKANISYNTH - WAVEFORM SynthesiS

波形をフリーハンドで描くことで音を生成するシンセサイザー・アプリケーション。視覚的形態と聴覚的出力を結びつける試み。

A synthesiser application that enables sound creation through freehand drawing of waveforms, linking visual form and auditory output.

RESEARCH PAPER

  1. Nakanishi, K., Haimes, P., Baba, T., & Kushiyama, K. (2016). NAKANISYNTH: An intuitive freehand drawing waveform synthesiser application for iOS devices. In Proceedings of the 16th International Conference on New Interfaces for Musical Expression (NIME 2016) (pp. 143–145). https://www.nime.org/proceedings/2016/nime2016_paper0029.pdf

TAIFŪRIN

日本的美意識に着想を得たアンビエント・メディア・システム。風鈴を用いて台風データを直感的に伝える試み。

An ambient media system, inspired by Japanese aesthetics, that uses a wind-chime installation to translate typhoon data into an intuitive, experiential form of risk communication. The name is a portmanteau of the Japanese words for "typhoon" and “wind chime”.

RESEARCH PAPERs

  1. Haimes, P., Baba, T., & Kushiyama, K. (2016). Taifūrin: Wind-chime installation as a novel typhoon early warning system. In Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology (Article 24, pp. 1–6). Association for Computing Machinery. https://doi.org/10.1145/3001773.3001830

  2. Kosmala, K., Clifford, A., Truslove, G., Bell, A., Grace, A., McGeechan, P., Scott, D., Jamieson, G., Holmes, P., Baba, T., McMeekin, R., Sherlock, S., Haimes, P., Kushiyama, K., Kanai, T., Kikukawa, Y., Kotegawa, T., Suda, H., Suzuki, T., ... Kushiyama, K. (2016). Azimuth: Sound/Image: Scottish version. Exhibition, CCA Glasgow. http://www.cca-glasgow.com/programme/uwsazimuth

BRAILLE PAD PROJECT

触覚および聴覚フィードバックを提供するタブレット型点字学習支援システム。指の動きと音を連動させることで、初期段階の日本語点字リテラシー向上を支援する。

A tablet-based Braille education support system that provides auditory and tactile feedback to help learners actively touch and trace Braille, linking finger movement with sound to enhance early Japanese Braille (Tenji) literacy.

RESEARCH PAPER

  1. Watanabe, S., Haimes, P., Baba, T., & Kushiyama, K. (2016). Braille Pad Project: Proposal of a Braille education support system using a tablet device. International Journal of Asia Digital Art and Design Association, 20(2), 35–40. https://www.jstage.jst.go.jp/article/adada/20/2/20_35/_pdf

Social robot design

社会的ロボットにあえて不完全さを取り入れ、人間の弱さや心に寄り添う関係のあり方を探る研究。

This work argues that designing social robots with imperfection can promote deeper, more ethical, and more meaningful human-robot relationships by resonating with human vulnerability, spirituality, and lived experience rather than machine-like perfection.

Research papers

  1. Rousi, R., & Haimes, P. (2024). Social robot design and the aesthetics of imperfection. In R. Rousi, C. von Koskull, & V. Roto (Eds.), Humane autonomous technology: Re-thinking experience with and in intelligent systems (pp. 199-232). Springer. https://doi.org/10.1007/978-3-031-66528-8_9

  2. Vakkuri, V., & Haimes, P. (2024). Bringing human-machine play to the stage. Creative Robotics Theatre Workshop, ACM Designing Interactive Systems Conference (DIS 2024), Copenhagen, Denmark. https://hoomansamani.com/creative-robotics/creative-robotic-theatre/creative-robotics-theatre-dis-24/

CC SOUNDTRACKER

音楽情報検索（MIR）の機械学習技術を用いて、動画の音声トラックを著作権フリーのCreative Commons（CC）音楽に自動置換するアプリケーション。

An application to replace a video’s soundtrack with copyright-free Creative Commons (CC) music using machine learning techniques from Music Information Retrieval (MIR).

RESEARCH PAPER

  1. What happened to this paper is a long and strange story… https://www.wired.com/story/steve-bannon-gaming-conference-saga/

HAZARD MAPPING

オーストラリアおよび日本における災害情報のアクセシビリティおよび視覚的訴求力の向上を目的としたプロジェクト。

Projects that aimed to increase the accessibility and visual appeal of hazard information in Australia and Japan.

Research papers

  1. Haimes, P., Baba, T., & Kushiyama, K. (2016). Katsukazan: An intuitive iOS app for informing people about volcanic activity in Japan. In Proceedings of the 2016 Symposium on Spatial User Interaction (p. 193). Association for Computing Machinery. https://doi.org/10.1145/2983310.2989191

  2. Haimes, P., Baba, T., & Medley, S. (2015). Mobile map applications and the democratisation of hazard information. In Proceedings of SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications (Article 7, pp. 1–4). Association for Computing Machinery. https://doi.org/10.1145/2818427.2818440

DIGITAL SONG-BIRD

人が机に近づくと、デジタルな鳥のさえずり音で通知するモーションセンサー搭載のシンプルなデバイス。研究論文は発表していない。

A simple device based on a motion sensor, designed to notify me via a digital tweet sound when someone approached my desk.

This one I just created for fun. No research paper from this project.

SCENARIO-BASED DESIGN

ユーザーの直接的な意見が得られない状況で、シナリオベースド・デザインがプロジェクト開始や要件整理に役立つことを示す研究。

This work shows how scenario-based design methodology can be used as a bridge to “fill the gap” when user input isn't directly available, as a catalyst to “kick-start” a design project, and as a brainstorming communication tool among designers to “tease out” and establish requirements. Visualisation by Stuart Medley.

RESEARCH PAPER

  1. Haimes, P . , Jung, J. H., & Medley, S. (2013). Bridging the gap: scenario-based design as a solution for delayed access to users. In Proceedings of Australian Council of University Art and Design Schools (ACUADS). https://ro.ecu.edu.au/ecuworks2013/241