TGUI and Unicode

[ingar]

Hi all,

A general question:

Are there any ways of supporting Unicode in TGUI? I am hoping to be able to replace a Windows CE application with a TGUI app. But the Windows CE app is connecting to a server that returns UTF8 data.
And I cannot see how to TGUI can display those Japanese names...

Ingar

[texus]

TGUI uses sf::String on most places, which supports unicode, you just need to get the data into sf::String. If your data is UTF8 and stored in some array of "char" or "unsigned char", you should be able to use sf::String::fromUtf8(data.begin(), data.end()).

If you want to display special characters then you need to make sure to use a font that can render those characters. Maybe the default TGUI font can, but it is possible that it will just render squares in which case you need a different font.

[ingar]

Hi again,

I am still having problems with tgui/sfml and Unicode. I have a server returning UTF8 data in a character buffer.
The data may contain Asian characters.

I have previously received some very useful help here, so I am thinking I am doing things the correct way:

First I am creating a "tgui::Font" object from a Windows font file (arial.ttf or other).

Then I am doing "m_labShipname->getRenderer()->setFont(Font)", where "m_labShipname" is
a pointer to a Label field. This works, I can see that the text displays in the correct font.

From the server I am receiving this in a character buffer, named "buffer":

0x41, 0x42, 0xe6, 0x9d, 0xb1, 0xe4, 0xba, 0xac,0

In case you do not see what this is, , it is AB東京 in UTF8 Japanese (A and B in front of "TOKYO").

I am then doing:

std::string s = buffer;
sf::String str;

str=sf::String::fromUtf8(s.begin(),s.end());
m_labShipname->setText(str);


I finally check the content of my "sf::String str". This I do using the sf::String::Iterator.
The iterator is of type Uint32 (this is a bit strange, why not Uint16?).
And the content of my Unicode string is:

0x0041 , 0x0042, 0x6771, 0x4eac

This is correct. Unfortunately, it is displayed as AB[][].

When I receive another string with Norwegian UTF8 characters containing ÆØÃ...,
they display correctly.

What could be wrong? Perhaps tgui/sfml does  not support the Windows True Type fonts?
Or did I miss something?

My OS is Raspian Stretch. I see the same on VmWare X86 as well as on a native Raspberry Pi.

Ingar

Hi all,

A general question:

Are there any ways of supporting Unicode in TGUI? I am hoping to be able to replace a Windows CE application with a TGUI app. But the Windows CE app is connecting to a server that returns UTF8 data.
And I cannot see how to TGUI can display those Japanese names...

Ingar

[texus]

I don't see anything wrong with your steps.

Maybe storing the buffer in an std::string can be skipped if fromUtf8 supports a begin and end pointer (I'm not sure if it does), but it will work equally well when first stored in an std::string.

The type is Uint32 because it stores the string as UTF32. Two bytes wouldn't be enough to store ALL possible unicode characters.

If squares are displayed then it would mean that the chosen font (arial.ttf in this case) does not contain these japanese characters.

[ingar]

HI again,

Arial is one of the most commonly used fonts used by Windows, and does indeed support Japanese and other Asian languages.
I also tried Tahoma. This font also displays just about everything.

One thing I haven't tried is to use the font in an SFML only app I have (i.e. without any tgui components).

Ingar

I don't see anything wrong with your steps.

Maybe storing the buffer in an std::string can be skipped if fromUtf8 supports a begin and end pointer (I'm not sure if it does), but it will work equally well when first stored in an std::string.

The type is Uint32 because it stores the string as UTF32. Two bytes wouldn't be enough to store ALL possible unicode characters.

If squares are displayed then it would mean that the chosen font (arial.ttf in this case) does not contain these japanese characters.

[texus]

Based on what I can find, Arial itself doesn't support these characters. I just downloaded a font called "Arial Unicode MS" which is 22MB in size and the characters are being displayed correctly.

My code is as simple as this:

Code (cpp) Select Expand

const char* buffer = "\x41\x42\xe6\x9d\xb1\xe4\xba\xac";
text->setText(sf::String::fromUtf8(&buffer[0], &buffer[8]));

Edit: for some reason that I haven't figured out yet, it only works when you use "m_labShipname->getRenderer()->setFont(font)" like you are doing. When using tgui::setGlobalFont I still get squares.
Edit 2: That seemed to be because I need to keep the font alive which is passed to setGlobalFont. Once I did that it also works when setting a global font.

[ingar]

Thanks a lot. I will try that.

[ingar]

Worked!

But I wonder what Mikkesoft is doing. Because their font files are about 1Mb in size, and I have never
had problems to display Unicode Asian characters.

Ingar