Noriyuki Kojima

I am a first year CS Ph.D student at Cornell University, working with Yoav Artzi in the intersection of natural language processing and computer vision. I am also a scholar of Masason Foundation since 2017.

Previously, I was a Research Specialist at Princeton University in 2019 working with Jia Deng; I graduated from the University of Michigan in 2018 with B.S.E in Computer Science where I worked with Jia Deng, Rada Mihalcea and Dragomir Radev; I was a software engineer intern in Facebook Applied Machine Learning team in 2017 where I was supervised by Juan Pino.

Email:  /  Google Scholar  /  LinkedIn  /  Twitter

Ph.D. Research

There are three major topics I am interested in for my Ph.D. studies, and I tend to work on the intersection.

  1. (Natural) Language Generation
  2. Vision and Language, Robotics, and beyond
  3. NLP Human Feedback / Reinforcement Learning

Here are some papers ...

What is Learned in Visually Grounded Neural Syntax Acquisitiond
Noriyuki Kojima, Hadar Averbuch-Elor, Alexander Rush, Yoav Artzi
ACL (Short Paper), 2020

Answering the question of what does grounded unsupervised constituency parser Visually Grounded Neural Syntax Learner (Shi et al., 2019) learn.

Undergraduate Research

I was widely intersted in Computer Vision and NLP in the context of not only machine learning but also hand-coded algorithms. Some topics I worked on for research were ...

  1. Computer Vision: 3D-Reconstrcution, Mesh-Generation
  2. NLP: Neural Machine Translation, Specialized Token Embeddings
  3. Computer Vision NLP: Phrase Localization on images, Multi-modal QA system
  4. Computer Vision Robotics: Autnomous Indoor Navigation
Here are some papers ...

OASIS: A Large-Scale Dataset for Single Image 3D in the Wild
Weifeng Chen, Shengyi Qian, David Fan, Noriyuki Kojima, Max Hamilton, Jia Deng,
CVPR, 2020

Open Annotations of Single Image Surfaces (OASIS): a dataset for images in the wild with dense annotations of detailed 3D geometry.

Representing Movie Characters in Dialogues
Mahmoud Azab, Noriyuki Kojima, Jia Deng, Rada Mihalcea
CoNLL, 2019   (Oral Presentation)

Representing character names by adding social networks of discourse in embedding objectives.

To Learn or Not to Learn: Analyzing the Role of Learning for Navigation in Virtual Environments
Noriyuki Kojima, Jia Deng,
Preprint, 2019

Comparing learning-based methods and classical methods for navigation in virtual environments. Classical methods > learning-based methods in high-level, but learning-based methods learned useful regularities.

Speaker naming in movies
Mahmoud Azab, Mingzhe Wang, Max Smith, Noriyuki Kojima, Jia Deng, Rada Mihalcea
NAACL, 2018

A new model for speaker naming in movies that leverages visual, textual, and acoustic modalities in an unified optimization framework.

Structured matching for phrase localization
Mingzhe Wang, Mahmoud Azab, Noriyuki Kojima, Jia Deng, Rada Mihalcea
ECCV, 2016

Proposing structured matching of phrases and regions that encourages the semantic relations between phrases to agree with the visual relations.

Last updated: October 2019
The template was borrowed from Jon Barron's implementaion.