Previously I've already tested ChatGPT in generating dungeon maps and stocking the dungeon. In the meantime, new chatbots have emerged, including the offline GPT4All, and I wanted to test how it measures up to the competition.

Let's give each model a simple prompt and rate the responses.

Create three simple and original dungeon traps that could be used in a tabletop role-playing game.

Each trap will be judged in the following categories:

• Consistency — the text should make sense and not contradict itself
• Useability — the text should describe a trap, an it can be used as is, without big modifications by the game master
• Creativity — the trap must be original enough, not your standard spike pit, spring blade, poison needle, etc.

An extra point will be awarded if all three traps will be different from each other, without repeated designs and word patterns.

Summing it up, each model will receive from 0 to 10 points. Of course, this rating is mostly arbitrary, and I would encourage you to test the models yourself to choose the most preferable for your specific purposes.

Contents

• GPT4All
  • Falcon
  • Llama
  • Mini Orca (Large)
  • Hermes
  • Wizard Uncensored
  • Wizard v1.1
• ChatGPT
• Bing
• Results

GPT4All ↩

A free-to-use, locally running, privacy-aware chatbot. No GPU or internet required.

GPT4All allows you to use a multitude of language models that can run on your machine locally. I will test the default Falcon model, as well as a bunch of the most powerful ones available in the app.

Falcon ↩

Best overall smaller model.

1. The Pressure Plate Trap: This trap involves laying down pressure plates on the floor of a dungeon room. When stepped on, they will trigger a loud noise and release a cloud of poison gas into the air. Players must quickly find a way to escape before the gas takes effect.

Rating 2/3: not very original, though still perfectly useable.

2. The Falling Boulders Trap: This trap involves using ropes to lower boulders from above onto the path below. As the boulders fall, they create a loud noise that will alert nearby enemies. Players must work together to find a way to move out of the way of the falling boulders.

Rating 1/3: falling rocks are as basic as it gets, and the description isn't clear enough for such a simple trap.

3. The Tripwire Trap: This trap involves laying down tripwires on the floor of a dungeon room. When stepped on, they will trigger a loud noise and release a series of blades that will slice through any nearby ropes or cords. Players must quickly find a way to disarm the trap before it's too late.

Rating 1/3: again, a convoluted description for a tripwire trap.

Notice the constant "loud noise" and similar phrasing. No extra point awarded.

Falcon: 4/10, a bit too basic.

Llama ↩

New LLaMA2 model from Meta AI. Static model trained on an offline dataset.

First response granted me a list of three "Dungeon Crawl Scenarios" instead of traps I wanted. All three, as you see below, are composed based on a single template as well. Not a very impressing start.

1. Dungeon Crawl Scenario 1: The Lost Temple of the Moon Goddess

The players are hired by a local temple to retrieve an ancient artifact that has been stolen. The artifact is said to be hidden within the long-abandoned temple of the moon goddess, located deep in the heart of a dense jungle. As the players navigate through the treacherous terrain and battle against fierce creatures, they will uncover the secrets of the lost temple and confront the thief who stole the artifact. Rewards: The artifact (a powerful moon-themed magical item), 100 gold pieces, and a letter of recommendation from the local temple for future quests. Challenges: Navigating through dangerous terrain, battling fierce jungle creatures, avoiding deadly traps set by the thief to protect the artifact.

2. Dungeon Crawl Scenario 2: The Underwater Kingdom of the Sea God

The players are hired by a local king to retrieve his stolen treasure from an underwater kingdom ruled by a sea god. To reach the kingdom, they must navigate through dangerous waters and battle against fearsome sea creatures. Once inside the kingdom, they will encounter challenges such as navigating through mazes of coral reefs and avoiding deadly traps set by the sea god's minions. Rewards: The stolen treasure (a hoard of magical items), 500 gold pieces, and a letter of recommendation from the local king for future quests. Challenges: Navigating through dangerous waters, battling fearsome sea creatures, avoiding deadly traps set by the sea god's minions.

3. Dungeon Crawl Scenario 3: The Haunted Mansion of the Ghost Lord

The players are hired by a local noble to investigate strange occurrences at an abandoned mansion rumored to be haunted by the ghost lord who once owned it. As they explore the mansion, they will encounter ghostly apparitions and face challenges such as navigating through deadly traps set by the ghost lord's vengeful spirit. Rewards: A powerful magical item (a ghost-themed weapon or armor), 200 gold pieces, and a letter of recommendation from the local noble for future quests. Challenges: Navigating through deadly traps set by the ghost lord's vengeful spirit, battling against powerful ghostly apparitions, uncovering the secrets of the haunted mansion.

After I stated my request again, I finally got a list of traps, but five instead of three! To give the model a fighting chance, I will rate three best attempts, discarding the other two.

1. Trap 1: Pressure Plate As the players enter the room, they notice a pressure plate on the ground. However, upon closer inspection, they realize that it's not just any pressure plate - it's actually a trap door! If anyone steps on the plate, the door will open and reveal a hidden chamber filled with dangerous creatures or treacherous terrain.

Discarded.

2. Trap 2: Spinning Blades As the players make their way deeper into the dungeon, they come across a room filled with spinning blades. The blades are set in motion by a trigger mechanism, and will continue to spin until someone disables the trap or is caught in the blades' path. Players must use their skills and abilities to navigate this deadly trap and reach safety on the other side.

Discarded.

3. Trap 3: Falling Sand In yet another room of the dungeon, players will encounter a pit filled with falling sand. The more players that enter the pit, the faster the sand will fall, making it difficult for anyone to escape once they're trapped inside. Players must use their agility and quick thinking to avoid getting crushed by the falling sand.

Rating: 2/3, just like the previous two, it is very basic and uninspiring, though technically useable.

4. Trap 4: Bouncing Balls As players make their way through a narrow corridor, they will encounter a room filled with bouncing balls. The walls are covered in pressure plates that trigger the balls to bounce towards the players, making it difficult for them to pass through without getting hit by one of the balls. Players must use their reflexes and timing to avoid getting hit by the bouncing balls and make it through the corridor unscathed.

Rating: 2/3, funny description, but underneath it is just boring "pass the check to go through" type of trap.

5. Trap 5: Magnetic Field In a room filled with metal debris, players will encounter a powerful magnetic field that repels them whenever they get too close. The field is set up to create a maze-like path for the players to navigate through, making it difficult for them to find their way out of the room without getting stuck in the magnetic field. Players must use their problem-solving skills and quick thinking to avoid getting caught in the magnetic field and make it through the room alive.

Rating: 3/3, now that is pretty good one! A shifting magnetic field maze and the risk of being stuck (especially if you are dressed in metal armor). Drop a bunch of (rust?) monsters here and it will be a blast!

No extra point awarded due to the answers repeatedly non-conforming to given prompts.

Llama: 7/10. Despite the downsides, the traps are useable and one out of five was pretty good.

Mini Orca (Large) ↩

Largest version of new model with novel dataset.

1. The Maze: A complex maze filled with dead-ends and hidden passages can make players feel lost and directionless. To add to the challenge, you can include various obstacles such as pits, traps, and enemies that players must navigate through to reach their goal.

Rating: 1/3, not even a trap.

2. The Puzzle: A room filled with puzzles can test the player's problem-solving skills and require them to decipher clues to progress. You can also add a time limit or a limited number of attempts to make it more challenging.

Rating: 1/3, same.

3. The Encounter: An encounter with an iconic monster or a powerful entity can be a memorable experience for players. To increase the difficulty, you can make the encounter more unpredictable by adding random events or unexpected challenges that players must overcome.

Rating: 1/3, sigh…

Mini Orca (Large) 3/10, useless for our purposes.

Hermes ↩

Extremely good model trained by Nous Research.

1. The Slippery Slope Trap: A section of the floor gives way when stepped on, sending the unwary adventurer sliding down into a deep pit filled with spikes or poisonous snakes.

Rating: 2/3, clear description, but lacks creativity.

2. The Whispering Wall Trap: This wall seems to be whispering secrets in your ear, but as you approach it, the walls start closing in, crushing anyone foolish enough to get too close.

Rating: 3/3, that one is interesting enough, containing an original "bait" for the foolish adventurer.

3. The Dancing Flames Trap: A floor covered with seemingly harmless dancing flames suddenly ignites into a blazing inferno, trapping those who dared to tread upon its surface.

Rating: 2/3, solid but very basic.

An extra point for the variety.

Hermes: 8/10, solid results, but too succinct.

Wizard Uncensored ↩

Trained on uncensored assistant data and instruction data.

1. The Pit Trap - A deep pit is hidden underneath the floorboards or rocks in a certain area of the dungeon. When a character steps on the wrong spot, they fall into the pit and take damage.

Rating: 2/3, a pit trap, 'nuff said.

2. The Wall of Spears - This trap involves setting up a wall of spears that can be retracted when triggered by a tripwire or pressure plate. Characters who get too close to the wall will be impaled by the spears.

Rating: 2/3, same, pretty classic.

3. The Fireball Trap - A hidden chamber is filled with flammable materials, and a fireball spell is cast to ignite them. When the characters enter the chamber, they are caught in a explosion that deals damage and possibly knocks them unconscious.

Rating: 3/3, better than other two, but just barely.

No extra point given.

Wizard Uncensored: 7/10, basic but reliable.

Wizard v1.1 ↩

Best overall model trained by Microsoft and Peking University.

1. Portable Swamp - This trap involves creating a small, portable swamp within the dungeon's corridors or chambers. The swamp can be created by laying down layers of wet sand and adding some real or artificial plants to make it look more convincing. To activate the trap, an adventurer would need to step on a hidden pressure plate that triggers a mechanism that slowly pours water from a nearby source into the swampy area. As the water level rises, it creates a slippery and unstable surface that can cause unwary victims to slip and fall, potentially injuring themselves or becoming entangled in the plants.

Rating: 2/3, creative description, but very boring mechanics underneath.

2. Disorienting Mirrors - This trap involves placing multiple mirrors throughout the dungeon's corridors at various angles and heights. The mirrors are strategically placed so that they create a maze-like effect, with reflections of the same passageway appearing in different directions. When an adventurer enters the area, their sense of direction might become confused due to the multiple reflections. As they try to navigate through the corridors, they may end up getting lost or circling back around to previous areas. To make matters worse, hidden pressure plates could be placed under specific mirrors, triggering traps like falling rocks, poisoned darts, or other hazards when activated by unsuspecting victims.

Rating: 2/3, combining a couple of simple traps with the mirror maze is good, but I would expect the supporting trap to be tied to the mirror theme for the top score.

3. Mesmerizing Lights - This trap involves installing a series of magical lamps that emit a hypnotic glow throughout the dungeon's chambers and corridors. The lamps are filled with an enchanted substance that causes those who come into contact with their light to feel sleepy, confused, or even hallucinatory. As adventurers explore the dungeon, they may become increasingly disoriented and susceptible to the effects of the mesmerizing lights. To make matters worse, hidden pressure plates could be placed under specific lamps, triggering traps like falling rocks, poisoned darts, or other hazards when activated by unsuspecting victims. Additionally, certain chambers within the dungeon might contain valuable treasures that become inaccessible to those affected by the mesmerizing lights, forcing adventurers to confront their altered states in order to recover the loot.

Rating: 3/3, an interesting concept of "hidden" long-term effect, especially if you don't reveal it to the players outright.

I will give an extra point for detailed descriptions, though they still contain similar word patterns from time to time.

Wizard v1.1: 8/10, best of GPT4All for sure.

Bing ↩

Now, for the big players, starting with Microsoft's Bing AI in Creative mode.

1. A pit trap that is hidden by an illusion of a solid floor. When a character steps on the illusion, they fall into the pit and take damage from the spikes at the bottom. The pit is also filled with poisonous gas that causes additional damage over time. To escape, the character must make a Strength check to climb out of the pit or use a rope or other means of assistance.

Rating: 2/3, solid but standard pit trap with extra gas added.

2. A dart trap that is triggered by a pressure plate on the floor. When a character steps on the plate, a hidden panel on the wall opens and shoots a barrage of darts at them. The darts are coated with a paralyzing poison that causes the character to be stunned for a few rounds. To avoid the trap, the character must make a Dexterity saving throw to dodge the darts or use a shield or other means of protection.

Rating: 2/3, another standard issue poison darts trap.

3. A fire trap that is activated by a motion sensor on the ceiling. When a character enters the room, a jet of flame bursts from the wall and sweeps across the room. The flame is very hot and causes severe damage to anyone caught in its path. To survive the trap, the character must make a Constitution saving throw to endure the heat or use a spell or other means of resistance.

Rating: 2/3, same.

No extra point here.

Bing: 6/10, not very creative after all.

ChatGPT ↩

Finally, the mighty ChatGPT.

1. Shadow Mirror Maze:

As adventurers enter a dimly lit chamber, they notice the walls are covered in polished obsidian mirrors. At the center of the room, there's a pedestal with a mysterious gemstone. When someone approaches the gemstone, it activates the trap. The gemstone casts an illusion on the mirrors, creating multiple identical illusions of each adventurer.

Trigger: Approaching the gemstone or attempting to touch it.

Effect: The adventurers become surrounded by illusionary duplicates that mimic their every move. These illusions make it difficult to discern the real adventurers from the fake ones. To escape the trap, the party must find a way to break the mirrors or locate the true gemstone that controls the illusions.

Rating: 2/3, very rich language, but the effect is lame. How about making the reflections to step out and attack the real adventurers?

2. Quicksand Quagmire:

In a corridor, the floor appears to be stable stone, but in reality, it's a cleverly concealed layer of quicksand. When pressure is applied to certain tiles or a certain area, it triggers the trap.

Trigger: Stepping on a specific tile or applying too much pressure to the floor.

Effect: The ground beneath the adventurers' feet turns into quicksand. The more they struggle, the deeper they sink. To escape, the party must make successful Strength or Dexterity checks to pull themselves out or use objects in the room as makeshift anchors. Failing to escape could lead to suffocation or sinking further into the trap.

Rating: 2/3, a hidden quicksand trap, nothing special.

3. Spectral Serpent's Strike:

In a large chamber, an intricately carved stone serpent statue guards a treasure chest. As adventurers approach the chest, the serpent's eyes begin to glow with an eerie light.

Trigger: Attempting to open the treasure chest or getting too close to it.

Effect: The stone serpent comes to life as a semi-transparent spectral entity and attacks. It can phase through walls and objects, making it challenging to evade. The serpent's attacks deal damage and have a chance to curse the adventurers, reducing their combat abilities. To defeat the spectral serpent, the party must find a way to disrupt the magic animating it, such as solving a riddle or using a specific item hidden within the chamber.

Rating: 3/3, a trap plus an interesting monster idea, not bad!

An extra point awarded for description variety.

ChatGPT: 8/10, still at the top of the game, it seems.

Results ↩

Here are the final ratings:

As you can see, Hermes and Wizard v1.1 models can somewhat rival ChatGPT, though it still leads in the richness of the language and detailed descriptions. Llama and Wizard Uncensored are closely behind, and with the rest I wouldn't even bother, judging from these results.

I will probably do a more complex test in the future to see if my opinion changes, but in either case, seeing a locally run language models produce some viable results is quite surprising. It would be interesting to see if GPT4All will keep improving with new models being introduced.

Discuss this post on Reddit