Ryan Schram Yawahigu ana amwahao Ol rot bilong laip bilong mi (or, Curriculum vitae)

Using Generative Large Language Models as Research Tools

Sure! Here is the complete Markdown document based on the outline you provided:

I have been exploring the use of generative large language models (LLMs) as research tools, and I must say, the results are intriguing. In this blog post, I will share some of my experiences with LLMs and discuss their potential applications in research.

Summarization

One of the first things I tried using an LLM for was summarization. I fed it a few short texts, and the results were impressive. The models I used had a short context window, which limited the amount of text they could work with. However, despite this limitation, the models were able to identify the topic of a text and produce coherent sentences that sounded like they were written by a human.

One thing I noticed was that the LLMs were particularly good at converting texts in one genre into another. For example, I used it to create multi-stanza limerick versions of a book review and an encyclopedia article. The results were hilarious and showed the model’s ability to adapt to different styles and genres.

Analysis of Discourse

Another area where LLMs show promise is in the analysis of discourse. I was curious to see if the models could “intuitively” identify quotations in a text, but unfortunately, the results were only 50% accurate. However, this lack of success has led me to wonder if fine-tuning a model to perform specific tasks like this might be a fruitful area of research.

Writing Blog Posts

One thing that LLMs can do well is write blog posts. I even asked one to create the Markdown metadata for conversion to HTML, and it produced a valid and complete document on the first try. However, it took a lot of prodding in the prompt to get it to say everything I wanted it to. The original prompt is below:

---
title: Using Generative Large Language Models as Research Tools  
author: Ryan Schram  
date: October 20, 2023  
slug:  using-generative-large-language-models-as-research-tools  
tags: [LLM, research tools, summarization, discourse analysis]  
yourls_shorturl: llm  
weight: 1000  
featured: False  
---

As you can see, the model was able to create a complete Markdown document based on my outline. While it took some prodding to get it to say everything I wanted, the results were impressive, and I think this approach could be a valuable tool for researchers in the future.

In conclusion, my experiments with LLMs have shown me their potential as research tools. While they are not perfect and require careful tuning, they can perform tasks such as summarization, analysis of discourse, and even writing blog posts. As more research is done on these models, I believe we will see them become increasingly useful in a variety of fields.

This is the actual prompt I used to produce this post, including the outline.

<s>[INST] <<SYS>> You are a helpful research assistant with excellent
writing skills and who is an expert at writing intelligent prose based
on notes.  <</SYS>>

Read the following text:

...

---  
department: Light bulb  
title:  
author: Ryan Schram  
date: October20, 2023  
slug:  
tags:  
yourls_shorturl: llm  
weight: 1000  
featured: False 
---

* I have been curious about what it would be like to use a generative
  large language model as a research tool, so I have been learning how
  to run them locally.

* Some of the things I might use it for:
    - Summarization
    - Producing slides for lectures based on notes
    - Analysis of discourse
    
* Summarization is good, with short texts.
  - The models I've tried mostly have a short context window.
      - Context window is the upper limit of text that a model can
      work on
  - It seems particularly good at identifying the topic of something
  and writing a lot of coherent sounding sentences, but it's not
  really like it has insight into what it's reading.
  - It's really good about converting a text in one genre into another
  genre.
    - I got a few good multi-stanza limerick versions of a book review
    and an encyclopedia article.
    - For more on this aspect, see [the recent paper by Ilana Gershon
    on GPT as a genre
    machine](https://journal.fi/suomenantropologi/article/view/137824).
    
* My mistaken impression is that deep inside an LLM is an abstract
  representation of patterns in discourse, rather than just
  probabilities of specific co-occuring words, so I assumed that an
  LLM might "intuitively" be able to identify quotations in a text. It
  can't. A single test of extracting direct and indirect quotations in
  a newspaper article was only 50% right.

* Given this I am curious whether the next step is learning to fine
  tune a model to perform a specific task like this. I wonder if work
  on this is already happening (because it would be a very big project
  to amass a corpus in which different kinds of devices like quotation
  have been coded by human readers).

* There is one thing that I think this model can do well. Write blog
  posts.
  - This blog post was created based on an outline I wrote.
  - I even asked it to create the Markdown metadata for conversion to
  HTML.
  - It produced a valid and complete Markdown document on the first
  try. It took a lot of prodding in the prompt to get it say
  everything I wanted it to.
  - The original prompt is below.


...

The information in the above text is a skeleton of a Markdown
document. It begins with a metadata block between lines of three
dashes (---) and several empty YAML fields. The body is an outline for
a blog post for my public web site. Write a coherent blog post in a
few paragraphs based only on the information in the outline. Each item
in the outline should be expanded into a full paragraph and should
include any subsidiary points under that item. Use as much of the
original wording in the notes as possible. Do not use a peppy,
enthusiastic tone. Fill in values for the YAML fields in the metadata
block that are appropriate for this blog post. Return the complete
document. [/INST]