PG Software Development

Hi Clifton,

One of the problems I have encountered recently involves encodeURIComponent. I use XMLHttpRequest to read files on the server. I thought that the path for the request should be encoded with encodeURIComponent. Maybe this is wrong because it seems that if there is a period in the name of the file to read, encodeURIComponent renders the path illegible.

I have made a small test app that I can send you if necessary. https://www.mediacours.com/programs/testencode The button on the left does not use encodeURIComponent whereas the one on the right does. So, should I be using encodeURIComponent in this case or not, please?

John

encodeURICompnent() is not intended for paths to access your POST file.
The function is intended for use with ONLY the query string.

Example:

Suppose you have a POST like this:
../php/mywidget.php?path=../php&var1=23&var2=Hello my name is John

In the case above the query string is all of the stuff following the "?" and in order for it to send correctly to a web server special characters must be encoded. Once encoded, the above query would look like this and be safe for upload to a web server:
../php/mywidget.php?path%3D..%2Fphp%26var1%3D23%26var2%3DHello%20my%20name%20is%20John

Here are the actions I used to build this example:
Suppose we set up two fields; one is named "query"—which contains the raw POST—and one that is named "encodedQuery"—which will display the result of encoding the query portion of the string.

On click ...
    Execute Script pgSplitToArray( text of field "query", "?" ); store return value in tmp;
    Execute Script encodeURIComponent( tmp [1] ); store return value in tmp [1];
    Set text of field "encodedQuery" to tmp [0] & "?" & tmp [1]
end click

Clicking a button will split the POST into the path to file AND query string; then it will encode the query portion; and finally reassemble the POST into a web-friendly result for use with XMLHttpRequest().

Sorry, I don't understand all of this. What exactly do you mean by a POST? Are you saying that any text to be stored in a file on a web-server should always be encoded? I understand that I must not use encodeURIComponent when defining the path for a php file but I don't understand in what circumstances I should use it.

In your submitted example, you DO NOT need to use encodeURIComponent() because there is no query string.

However, if you DO INCLUDE a query string "? . . .", then you MUST USE encodeURIComponent() to encode the query or else sending the data to the web server from the client browser will fail IF it contains special characters like forward and backslashes, ampersands, sometimes spaces, etc.

Please see this reference for more information:
https://www.w3schools.com/tags/ref_urlencode.asp

Thanks for your reply. I understand your example better now, I think. The path to a php file should not be encoded but the parameter(s) to be used by the php file should be encoded. Using XMLHttpRequest the text for the actions editor section URL should not be encoded but the params_valuePairs section should be. Right?

One other thing, to which I think I know the answer but have doubts. If I upload a UTF 8 text file to the server containing special characters Comme ça peut-être the contents of the file does not have to be encoded. It can be uploaded and then read on the the server as it is? This seems to be the case but ...

Regarding the text file uploaded to the server, I would say it should be fine as it is in UTF-8. However, you may have to experiment with this and if your web server doesn't give back the characters to the client as you expect, you may have to search the web to figure out how to configure your web server, etc. to get things to synchronize.

I've had troubled with PayPal in this regard when they send IPN's back to my web server. If a client is making a purchase for something from certain countries, then sometimes the IPN from PayPal will not alway confirm because their name, street address, etc. may contain language characters which suddenly do not check out between my web server and PayPal's IPN server. Fortunately, this only happens once in a while, but I have yet to figure out why the servers end up communicating at UTF-8 on my end and PayPal at Windows-1252. Just haven't had time or enough trouble with this to figure it out yet. Maybe your experiments may reveal something that both of us may benefit from.

OK. This is what I think I have proved (using XAMPP and localhost):

- You can upload a UTF 8 text file containing non ASCII characters and then read it successfully from the server.
- You can create a UTF 8 text file containing non ASCII characters in a TB DHTML app in a browser and then read it successfully from the server.

I'm not sure that this will be much help to you but the files I used are here: https://www.mediacours.com/tb_examples/testEncode.zip

To use the app you must make a /programs/ folder on your server root and put the 3 php files in it. Paste the text in French into the large field before you click on 'write file'. I hope it works.

I think I may have found the solution to your PayPal problem. Unfortunately I can't find it in English. However, I have translated the final key section:

Under the 'My account' tab, click on 'Preferences'
In 'My sales tools' right at the bottom in 'More sales tools'
Click on 'Choice of language for the PayPal button'.

Now click on 'Other options'
Finally, select UTF 8

(Use the link below to the site in French. Follow the instructions above. Some of the key words may not be correct so look for synonyms. Good luck! )

http://www.michelpaquin.com/2012/12/changer-le-charset-windows-1252-pour-utf-8-dans-paypal/

I will definitely check this and see what setting mine is at.
Thank you!

Hi Clifton,

New information:

If I use XMLHttpRequest to read a file on the server directly i.e. the url parameter is the path + file of the file I want to read I have no problem.

If I use a php file containg the line echo file_get_contents($fileName) the text returned contains unknown character symbols.

I can make these disappear by using $String = mb_convert_encoding($String, 'HTML-ENTITIES', "UTF-8"); in the php file but for reasons unknown my program still can't use the text returned.

So it's not a problem because I use the first solution but I thought you might like to know about this.

John